NAME

GETTING STARTED

SYNOPSIS

DESCRIPTION

OPTIONS

PRUNING OPTIONS

IMPLEMENTATION OPTIONS

EXAMPLES

APPLICABILITY

FILES

ENVIRONMENT

DIAGNOSTICS

BUGS

AUTHOR

HISTORY/CREDITS

SEE ALSO

REFERENCES

NOTES

Sets the main inflation value to <num>. This value is the main handle for affecting cluster granularity. It is usually chosen somewhere in the range [1.2-5.0]. -I 5.0 will tend to result in fine-grained clusterings, and -I 1.2 will tend to result in very coarse grained clusterings. Your mileage will vary depending on the characteristics of your data. That is why it is a good idea to test the quality and coherency of your clusterings using clm dist and clm info. This will most likely reveal that certain values of -I are simply not right for your data. The clm dist section contains a discussion of how to use the cluster validation tools shipped with mcl (see the SEE ALSO section).

With low values for -I, like -I 1.2, you should be prepared to use more resources in order to maintain quality of clusterings, i.e. increase the argument to the -scheme option.

The default mode of output creation for mcl is to create a file name that uses the input file name stripped of any leading path components, augmented with a prefix 'out.' and a suffix encoding pivotal mcl parameters. This will usually be the inflation value which is the argument to the -I option. By default the output file is written in the current directory. For example, if the input is named data/small.mci for example and inflation is set to three, the output file will be named out.small.mci.I30.

This behaviour can be overridden in various ways. The -o option simply specifies the output file name, which may include path components that should exist. It is possible to send the clustering to STDOUT by supplying -o -. With the -odir <dname> option mcl constructs the output file name as before, but writes the file in the directory <dname>. Finally, the option --d is similar but more specific in that mcl will write the output in the directory specified by the path component of the input file, that is, the directory in which the input file resides.

If either one of --abc, --sif, --etc or -use-tab tab-file is used the output will be in label format. Otherwise the clustering is output in the mcl matrix format; see the mcxio section for more information on this. Refer also to the group of options discussed at --abc.

Look at the -ap prefix option and its siblings for the automatic naming constructions employed by mcl if the -o option is not used.

With the -c <num> option, as the final step of loop computation (i.e. after initialization and shadowing) all loop weights are multiplied by <num>, if supplied.

By default mcl will remove any loops that are present in the input. Use --discard-loops=n to turn this off. Bear in mind that loops will still be modified in all cases where the loop weight is not maximal among the list of edge weights for a given node.

These items all relate to label input and/or label output. --abc tells mcl to expect label input and output clusters in terms of those labels. This simple format expects two or three fields separated by white space on each line. The first and second fields are interpreted as labels specifying source and destination node respectively. The third field, if present, specifies the weight of the arc connecting the two nodes.

The option --sif tells mcl to expect SIF (Simple Interaction File) format. This format is line based. The first two fields specify the source node (as a label) and the relationship type. An arbitrary number of fields may follow, each containing a label identifying a destination node. The second field is simply ignored by mcl. As an extension to the SIF format weights may optionally follow the labels, separated from them with a colon character. It is in this case necessary to use the --expect-values option. The --etc option expects a format identical in all respects except that the relationship type is not present, so that all fields after the first are interpreted as destination labels.

-use-tab is only useful when matrix input is used. It will use the tab file to convert the output to labels; it does not fail on indices missing from the tab file, but will bind these to generated dummy labels.

-tf transforms the values of the input matrix according to <tf-spec>. -abc-tf transforms the stream values (when --abc is used) according to <tf-spec>. --abc-neg-log and --abc-neg-log10 imply that the stream input values are replaced by the negation of their log or log10 values, respectively. The reason for their existence is documented in mcxio. For a description of the transform language excpected/accepted in <tf-spec> refer to the same.

With this option mcl will subcluster the provided clustering. It does so by removing, first of all, all edges from the input graph that connect different clusters. The resulting graph consists of different components, at least as many as there are clusters in the input clustering. This graph is then subjected to transformations, if any are specified, and then clustered. The output name is constructed by appending the normal mcl-created file name suffix to the name of the input clustering.

The first two options are somewhat outdated, in that the prefered way of loading networks is by using mcxload. The option -write-expanded can be useful for exploring more complicated input transformations that incorporate an expansion step, but is not really relevant for production use. The last option is mainly educational and for analyzing the mcl process itself.

There are currently seven different resource schemes, indexed 1..7. High schemes result in more expensive computations that may possibly be more accurate. The default scheme is 4. When mcl is done, it will give a grade (the so called jury synopsis) to the appropriateness of the scheme used. A low grade does not necessarily imply that the resulting clustering is bad - but anyway, a low grade should be reason to try for a higher scheme.

Use the -resource <num> option to cap for each nodes the number of neighbours tracked during computation at <num> nodes.

The PRUNING OPTIONS section contains an elaborate description of the way mcl manages resources, should you be interested. In case you are worried about the validation of the resulting clusterings, the mclfaq section has several entries discussing this issue. The bottom line is that you have to compare the clusterings resulting from different schemes (and otherwise identical parameters) using utilities such as clm dist, clm info on the one hand, and your own sound judgment on the other hand.

If your input graph is extremely dense, with an average node degree (i.e. the number of neighbours per node) that is somewhere above 500, you may need to filter the input graph by removing edges, for example by using one of -tf '#ceilnb()' or -tf '#knn()'.

Shows the explicit settings to which the different preset schemes correspond.

The characteristics are written in the same format (more or less) as the output triggered by -v pruning.

See the -v option below.

These are the different verbosity modes:

pruning
explain
cls
all

To make mcl as quiet as can be, add -q x -V all to the command line.

The -q option governs a general logging mechanism. The format accepted is described in the tingea.log manual page.

The other options govern verbosity levels specific to mcl. -v all turns them all on, -V all turns them all off. -v str and -V str turn on/off the single mode str (for str equal to one of pruning, cls, or explain). Each verbosity mode is given its own entry below.

This mode causes the output of explanatory headers illuminating the output generated with the pruning verbosity mode.

This mode causes output of resource-related quantities. It has a separate entry in the PRUNING OPTIONS section.

This mode (on by default) prints a terse list of characteristics of the clusterings associated with intermediate iterands. The characteristics are E/V, cls, olap, and dd. They respectively stand for the number of outgoing arcs per node (as an average), the number of clusters in the overlapping clustering associated with the iterand, the number of nodes in overlap, and the dag depth associated with the DAG (directed acyclic graph) associated with the iterand. For more information on this DAG refer to the -dump option description in this manual and also mclfaq.

Standard log information

See the -ap option below.

If the -o fname option is not used, mcl will create a file name (for writing output to) that should uniquely characterize the important parameters used in the current invocation of mcl. The default format is out.fname.suf, where out is simply the literal string out, fname is the first argument containing the name of the file (with the graph) to be clustered, and where suf is the suffix encoding a set of parameters (described further below).

The -ap str option specifies a prefix to use rather than out.fname as sketched above. However, mcl will interpret the character '=', if present in str, as a placeholder for the input file name.

If the -aa str option is used, mcl will append str to the suffix suf created by itself. You can use this if you need to encode some extra information in the file name suffix.

The suffix is constructed as follows. The -I f and -scheme parameter are always encoded. Other options, such as -pi f and -knn are only encoded if they are used. Any real argument f is encoded using exactly one trailing digit behind the decimal separator (which itself is not written). The setting -I 3.14 is thus encoded as I31. The -scheme option is encoded using the letter 's', all other options mentioned here are encoded as themselves (stripped of the hyphen). For example

results in the file name out.small.mci.I30s5c25pi08. If you want to know beforehand what file name will be produced, use the -az option.

If mcl automatically constructs a file name, it can be helpful to known beforehand what that file name will be. Use -az and mcl will write the file name to STDOUT and exit. This can be used if mcl is integrated into other software for which the automatic creation of unique file names is convenient.

By default mcl incorporates the input file name into the output file name and appends a short suffix describing the most important option settings. Use -ax to find out what that suffix is. This can be useful in wrapper pipeline scripts such as clxcoarse.

mcl writes the command line with which it was invoked to the output clustering file. Use this option to include any additional information. MCL does nothing with this option except copying it as just described.

Threading is useful if you have a multi-processor system. mcl will spawn k threads of computation. If these are computed in parallel (this depends on the number of CPUs available to the mcl process) it will speed up the process accordingly.

When threading, it is best not to turn on pruning verbosity mode if you are letting mcl run unattended, unless you want to scrutinize its output later. This is because it makes mcl run somewhat slower, although the difference is not dramatic.

If used, mcl will apply inflation one time to the input graph before entering the main process. This can be useful for making the edge weights in a graph either more homogeneous (which may result in less granular clusterings) or more heterogeneous (which may result in more granular clusterings). Homogeneity is achieved for values <num> less than one, heterogeneity for values larger than one. Values to try are normally in the range [2.0,10.0].

The -ph option is special in that it does not rescale columns to be stochastic. Instead, it rescales columns so that the maximum value found in the column stays the same after inflation was applied. There is little significance to this, and what little there is is undocumented.

If used, mcl will apply inflation one time to the input graph before entering the main process. The difference with -pi is that with the latter option mcl may apply certain transformations after reading in the matrix such as adding or modifying loops. The purpose of the -if (mnemonic for inflation-first) option is to use it on graphs saved with the --write-expanded option and convey to mcl that it should not apply those transformations.

Dump during iterations i..j-1. Use all to dump in all iterations. See the -dump str option below.

Sampling rate: select only these iterations in the dump interval. See the -dump str option below.

Set the the stem for file names of dumped objects (default mcl). See the -dump str option below.

str is checked for substring occurrences of the following entries. Repeated use of -dump is also allowed.

ite
dag
cls
chr
lines
cat

lines and cat change the mode of dumping. The first changes the dump format to a line based pairwise format rather than the default mcl matrix format. The second causes all dumped items to be dumped to the default stream used for the output clustering, which is appended at the end.

The ite option writes mcl iterands to file. The cls option writes clusterings associated with mcl iterands to file. These clusters are obtained from a particular directed acyclic graph (abbreviated as DAG) associated with each iterand. The dag option writes that DAG to file. The DAG can optionally be further pruned and then again be interpreted as a clustering using clm imac, and clm imac can also work with the matrices written using the ite option. It should be noted that clusterings associated with intermediate iterands may contain overlap, which is interesting in many applications. For more information refer to mclfaq and the REFERENCES section below.

The result option dumps the usual MCL clustering.

The chr option says, for each iterand I, to output a matrix C with characteristics of I. C has the same number of columns as I. For each column k in C, row entry 0 is the diagonal or 'loop' value of column k in I after expansion and pruning, and before inflation and rescaling. Entry 1 is the loop value after inflation and rescaling. Entry 2 is the center of column k (the sum of its entries squared) computed after expansion and before pruning, entry 3 is the maximum value found in that column at the same time. Entry 4 is the amount of mass kept for that column after pruning.

The -ds option sets the stem for file names of dumped objects (default mcl). The -di and -dm options allow a selection of iterands to be made.

This has two completely different uses. It sets the number of decimals used for pretty-printing mcl iterands when using the --show option (see below), and it sets the number of decimals used for writing the expanded matrix when using the -write-expanded option.

Print matrices to screen. The number of significant digits to be printed can be tuned with -digits n. An 80-column screen allows graphs (matrices) of size up to 12(x12) to be printed with three digits precision (behind the comma), and of size up to 14(x14) with two digits. This can give you an idea of how mcl operates, and what the effect of pruning is. Use e.g. -S 6 for such a small graph and view the MCL matrix iterands with --show.

Write matrix dump output in binary mcl format rather than interchange mcl format (the default). Note that mcxconvert can be used to convert each one into the other. See mcxio and mcx for more information.

str can be one of lex, size, revsize, or none. The default is 'revsize', in which the largest clusters come first. If the mode is 'size', smallest clusters come first, if the mode is 'lex', clusters are ordered lexicographically, and if the mode is 'none', the order is the same as produced by the procedure used by mcl to map matrices onto clusterings.

Mode keep causes mcl to retain overlap should this improbable event occur. In theory, mcl may generate a clustering that contains overlap, although this almost never happens in practice, as it requires some particular type of symmetry to be present in the input graph (not just any symmetry will do). Mathematically speaking, this is a conjecture and not a theorem, but the present author wil eat his shoe if it fails to be true (for marzipan values of shoe). It is easy though to construct an input graph for which certain mcl settings result in overlap - for example a line graph on an odd number of nodes. The default is to excise overlapping parts and introduce them as clusters in their own right. It is possible to allocate nodes in overlap to the first cluster in which they occur (i.e. rather arbitrarily), corresponding with mode cut.

In mode split mcl will put all nodes in overlap into separate clusters. These clusters are chosen such that two nodes are put in the same new cluster if and only if they always occur paired in the clusters of the overlapping clustering.

This option has no effect on the clusterings that are output when using -dump cls - the default for those is that overlap is not touched, and this default can not yet be overridden.

If the input graph has strong bipartite characteristics, mcl may yield clusters that do not correspond to connected components in the input graph. Turn one of these modes on to analyze the resultant clustering.

If loose clusters are found they will be split into subclusters corresponding to connected components. With --force-connected=y mcl will write the corrected clustering to the normal output file, and the old clustering to the same file with suffix orig. With --check-connected=y mcl will write the loose clustering to the normal output file, and the corrected clustering to the same file with suffix coco.

These options are not on by default, as the analysis is currently (overly) time-consuming and mcl's behaviour actually makes some sense (when taking bipartite characteristics into account).

With this mode turned on, mcl will reread the input matrix and compute a few performance criteria and attach them to the output file. Off by default.

Shows the log with process characteristics on STDOUT. By default, this mode is off.

Explains how the jury synopsis is computed from the jury marks.

Show version.

<int> is interpreted as the number of nodes of an input graph. mcl will print the maximum amount of RAM it needs for its computations. The formula for this number in bytes is:

This estimate will usually be too pessimistic. It does assume though that the average node degree of the input graph does not exceed k. The -how-much-ram option takes other command-line arguments into account (such as -S and -R), and it expresses the amount of RAM in megabyte units.

Shows a selection of the most important mcl options.

Gives a one-line description for all options.

Show current settings for tunable parameters. --show-settings is a synonym.

After computing a new (column stochastic) matrix vector during expansion (which is matrix multiplication c.q. squaring), the vector is successively exposed to different pruning strategies. The intent of pruning is that many small entries are removed while retaining much of the stochastic mass of the original vector. After pruning, vectors are rescaled to be stochastic again. MCL iterands are theoretically known to be sparse in a weighted sense, and this manoever effectively perturbs the MCL process a little in order to obtain matrices that are genuinely sparse, thus keeping the computation tractable. An example of monitoring pruning can be found in the discussion of -v pruning at the end of this section.

mcl proceeds as follows. First, entries that are smaller than cutoff are removed, resulting in a vector with at most 1/cutoff entries. The cutoff can be supplied either by -p, or as the inverse value by -P. The latter is more intuitive, if your intuition is like mine (P stands for precision or pruning). The cutoff just described is rigid; it is the same for all vectors. The --adapt option causes the computation of a cutoff that depends on a vector's homogeneity properties, and this option may or may not speed up mcl.

Second, if the remaining stochastic mass (i.e. the sum of all remaining entries) is less than <pct>/100 and the number of remaining entries is less than <r> (as specified by the -R flag), mcl will try to regain ground by recovering the largest discarded entries. The total number of entries is not allowed to grow larger than <r>. If recovery was not necessary, mcl tries to prune the vector further down to at most s entries (if applicable), as specified by the -S flag. If this results in a vector that satisfies the recovery condition then recovery is attempted, exactly as described above. The latter will not occur of course if <r> <= <s>.

The default setting is something like -P 4000 -S 500 -R 600. Check the -z flag to be sure. There is a set of precomposed settings, which can be triggered with the -scheme k option. k=4 is the default scheme; higher values for k result in costlier and more accurate computations (vice versa for lower, cheaper, and less accurate). The schemes are listed using the --show-schemes option. It is advisable to use the -scheme option only in interactive mode, and to use the explicit expressions when doing batch processing. The reason is that there is no guarantee whatsoever that the schemes will not change between different releases. This is because the scheme options should reflect good general purpose settings, and it may become appararent that other schemes are better.

Note that 'less accurate' or 'more accurate' computations may still generate the same output clusterings. Use clm dist to compare output clusterings for different resource parameters. Refer to clm dist for a discussion of this issue.

The two options -warn-pct and -warn-factor relate to warnings that may be triggered once the initial pruning of a vector is completed. The idea is to issue warnings if initial pruning almost completely destroys a computed vector, as this may be a sign that the pruning parameters should be changed. It depends on the mass remaining after initial pruning whether a warning will be issued. If that mass is less than warn-pct or if the number of remaining entries is smaller by a factor warn-factor than both the number of entries originally computed and the recovery number, in that case, mcl will issue a warning.

-warn-pct takes an integer between 0 and 100 as parameter, -warn-factor takes a real positive number. They default to something like 30 and 50.0. If you want to see less warnings, decrease warn-pct and increase warn-factor. Set warn-factor to zero if you want no warnings.

Pruning verbosity mode causes mcl to emit several statistics related to the pruning process, each of which is described below. Use -v explain to get explanatory headers in the output as well (or simply use -v all).

This value (by default set to 10) determines when mcl switches to sparse matrix/vector multiplication. For a given column stochastic vector (corresponding with all the neighbours of a given node v according to the current mcl iterand) the sum S of neighbour counts of all neighbours of v is computed, counting duplicates. This is exactly the number of matrix entries involved in the computation of the new column vector for the matrix product. If S times <int> does not exceed the number of nodes in the graph (equal to both column and row dimension of the matrices used) then a sparse implementation is used. Otherwise an optimized regular implementation is used. Intuitively, this option can be thought of as the estimated overhead per matrix floating point operation incurred by the sparse implementation compared with the regular implementation. MCL uses this estimated overhead to determine which implementation is likely to be quicker. Testing has shown this strategy works very well for graphs of a wide range of sizes, including graphs with up to 3 million nodes and 500 million edges.

NOTE
The effectiveness of this option is influenced by hardware-specific properties such as the CPU L2 cache size. The default value should work reasonably well across a wide variety of scenarios, but it may be possible to squeeze faster run times out of mcl by tuning this parameter to the graphs that are specific for your application domain.

When writing matrices in interchange format, mcl will use this variable (if present) as the precision (number of digits) for printing the fractional part of values.

MCL and its sibling applications will usually report about matrix input/output from/to disk. The verbosity level can be regulated via MCLXIOVERBOSITY. These are the levels it can currently be set to.

MCL and its sibling applications will by default output matrices in interchange format rather than binary format (cf. mcxio). The desired format can be controlled via the variable MCLXIOFORMAT. These are the levels it can currently be set to.

If matrices are output in interchange format, by default empty vectors will not be listed. Equivalently (during input time), vectors for which no listing is present are understood to be empty - note that the presence of a vector is established using the domain information found in the header part. It is possible to enforce listing of empty vectors by setting bit '1' in the variable MCLXICFLAGS.

MCL and its sibling applications will always check a matrix for consistency while it is being read. If this variable is set, the consistency check is omitted. For large graphs the speed up can be considerable. However, if the input graph is not conforming it will likely crash the application that is using it.