HeatmapR 2.0.0 is a major release: a new layout that sizes every part of a heatmap from its contents, clustering in compiled code that scales to hundreds of thousands of rows, many new kinds of heatmap (annotations, split cells, distributions inside cells, zoom panels, correlation heatmaps, branch support and more), and heatmaps, dot plots, markers and trajectories of single-cell, sequencing and cytometry data. Some conventions and defaults have changed; they are listed first.
hclust()), Hartigan-Wong k-means (from kmeans()), the dip (from the diptest package), and distances, correlations, quantiles, means and colour ramps written to match R's arithmetic - has been rewritten from the published algorithms and R's documented behaviour, and checked against R's functions run as black boxes. Results are the same, up to rounding in the last digits, except where distances are exactly tied: centroid and median linkage now take tied pairs in the order of the clusters' first leaves, which hclust() follows in most but not all cases, so the trees of tied data can differ from earlier versions (both are valid).heat_map() has a new layout. Dendrograms, bar graphs, annotation tracks, cluster label strips, the legend and the margins are sized in inches from their contents, and the heatmap body gets the remaining space. Dendrograms and bar graphs are 0.6 inches (times tree_size_* and bar_size_*) whatever the number of rows and columns, the legend is as wide as its text and stacked from the top of the heatmap (legend_size scales the legend keys), axis labels and the title sit just beyond the axis text, and margin text is kept on the device. The layout of HeatmapR 1.1.0 and earlier, in which these components were sized as fractions of the heatmap, has been removed.x is at the top of a heatmap without a row dendrogram (it was at the bottom, so heatmaps were upside down compared with the printed data), and the first leaf of a row dendrogram (heat_map_info()$tree_y$order[1], as for hclust and heat_map_clust()) is at the top. Row orders, clusters and dendrograms are now read the same way as those of the columns, so heat_map_info()$row_order equals the dendrogram order and, without a dendrogram, the order of the data. Clustered heatmaps are mirrored top to bottom compared with earlier versions.tree_cut_x/tree_cut_y are read in the order the heatmap is shown without them (the dendrogram order when a dendrogram is requested), so clusters can be assigned by reading a heatmap. Cluster sizes, axis_text_x/axis_text_y, tree_label_text_x/tree_label_text_y and the other settings with one value per row, column or cluster, and numeric positions in mark_x/mark_y and zoom_x/zoom_y follow the same order. Rows are now read from the top: axis_text_y, row cluster sizes and labels were given from the bottom, and row clusters are now numbered from the top (also in heat_map_info()$row_clusters), with the first cluster drawn at the top. Values that belong to the data are still matched to it: cluster labels given as character or factor vectors (e.g. from metadata), named indices, logical mark_*/zoom_* vectors, bar_values_*, annotations and cell matrices follow the original order of x or its names.x, which are regrouped so that each cluster is contiguous. If a dendrogram is also requested, the rows/columns are clustered within each cluster and the clusters are joined at the top of the dendrogram (see New features); a dendrogram supplied as an hclust or dist object is kept only when the clusters are contiguous in its order, and dropped with a warning otherwise.heat_map_scale() now errors for unsupported scaling methods (e.g. "minmax" was previously treated as "mean") or margins, instead of silently guessing. The margin must be "column" or "row" (or an abbreviation such as "col"), or "x"/"y"; previously any value not starting with "c", including "x", scaled rows.heat_map(scale = "range") (or "mean"/"zscore") now applies that scaling method to each column. Previously the method name was treated as the margin, so rows were scaled (and scrambled by the row scaling bug below).cell_col_scale to a vector of colours, a palette name or a function for other scales.clust_max rows or columns (5000 by default; max in heat_map_clust()) are clustered from a sample of clust_max of them rather than exactly, which keeps the memory and time needed in check (see New features). Set clust_max = Inf (or max = Inf) to cluster exactly, as earlier versions did.stats::hclust() gives. Complete, average, McQuitty and Ward linkage use the nearest-neighbour chain algorithm, which gives the same tree as stats::hclust() when distances are distinct. When distances are tied (common for integer, binary or rounded data) more than one tree is correct, and the tree can differ from the one stats::hclust() and earlier versions return while being equally valid.stats::dist() and cor(use = "pairwise.complete.obs") for every distance: Pearson and cosine distances compare the values each pair of rows shares (Pearson correlations are computed as cor() computes them).tree_split_x, tree_split_y) are 0.08 inches per unit, so they stay visible on large heatmaps; they were half a row or column, which vanished with thousands of rows.heat_map() returns a heat_map object (still a recordedplot) that can be redrawn with print()/plot() (with optional argument overrides) and saved with heat_map_save(file, x = hm).popup = TRUE, the default of heat_map() and heat_map_custom()) are only opened in interactive sessions: scripts, R Markdown and other non-interactive sessions draw on the current device instead of opening a new file device for every heatmap.heat_map() leaves the heatmap's margins set on the device, so base graphics drawn afterwards (e.g. points() or text() at the cell centres from heat_map_data()) land on the heatmap and aren't clipped; previously the margins were reset after drawing.tree, distance and linkage names, flags, sizes and label lengths, fractional mark_* positions and cluster numbers, malformed hclust objects, and annotation or bar value names that match only some of the rows or columns. Arguments passed through ... that neither dist() nor hclust() use give a warning, catching misspelled argument names that were previously ignored.legend = TRUE now draws the "levels" legend whenever x has non-numeric columns (legend = "both" leaves it out, as TRUE did). The values take the colours of cell_col_palette in the order of their factor levels, or sorted, as annotations do (they were taken in the order the values first appear).hcl.colors()), and installing from source needs a C++ compiler: the package now contains C++ code.annotation_x and annotation_y take a data.frame describing the columns or rows of x (matched by name when it has row names), drawn as coloured tracks beside the heatmap with their names and legends. Non-numeric annotations are coloured by level and numeric annotations with a colour ramp; annotation_colours sets colours per annotation (named vectors, palettes or functions). annotation_size_*, annotation_text_* and annotation_legend control their appearance.cell_col_palette, non-numeric values beyond the six default colours take distinct colours of several lightnesses, each the furthest in CIELAB from the colours before it and from white (as Glasbey et al. 2007 choose them), instead of colours interpolated between the six; annotations with more than 12 levels and stacked bars of more than 12 parts keep the first eight colours of their palette and continue the same way, so that 30 clusters can be told apart. The colours are the same each time, and palettes that are given are used as before.tree_col_x and tree_col_y colour the branches of a dendrogram by cluster - one colour for each cluster of tree_cut_x/tree_cut_y (or k-means), from left to right and from top to bottom, colouring the branches within each cluster and the edge into it while the branches joining clusters stay black (as dendextend's color_branches()), TRUE for distinct colours, or one colour for the whole tree. The cluster labels take the colours of their branches unless tree_label_col_x/tree_label_col_y is given, and heat_map_data(hm, "tree_y") gives the cluster below each segment.tree_cut_x/tree_cut_y is a factor or character vector and cluster labels are shown, the clusters are labelled with their levels by default.legend_side = "bottom" draws the legends below the heatmap, side by side with horizontal colour bars; cell_col_scale_limits = "robust" uses the 2nd and 98th percentiles so that outliers don't wash out the colour scale; cell_col_scale_breaks bins values into a stepped colour scale (as pheatmap's breaks); and legend_col_breaks sets the values (and, with names, the labels) of the colour legend ticks.dist_method (and heat_map_clust(dist = )) accepts "pearson", "spearman" and "kendall" (one minus the correlation; rows without variation are treated as uncorrelated) or a function returning a dist object or square matrix. Distance and linkage names ignore case and can be abbreviated, as stats::dist() and stats::hclust() accept them.tree_order_x/tree_order_y (and heat_map_clust(order = )) flip dendrogram branches with "olo" (optimal leaf ordering, minimising the distances between neighbouring rows/columns; exact, without new dependencies) or "mean" (branches ordered by their means; stats::heatmap() orders them by the sum of their leaves' means). Both also apply within groups.tree_order_x/tree_order_y = "spectral" (and heat_map_clust(order = "spectral")) order rows or columns by recursive spectral bisection of the graph joining each to its 6 nearest neighbours (weighted from the nearest one, as in UMAP, so that the distance every pair of rows shares in many dimensions doesn't flatten the weights): every part is sorted by its Fiedler vector, split where the graph is thinnest and the halves turned to meet where they are closest, and runs of up to 12 rows are then reversed where that shortens the path through them (windowed 2-opt). Gradients, trajectories and time courses are laid out along their length, cycles around their loop and clusters one after another, which a dendrogram can only approximate (its orders flip whole branches). A spectral order needs no dendrogram - rows can be ordered on their own, within groups or k-means clusters - and with one the branches are ordered by the mean spectral position of their leaves. On test data (a trajectory, clusters, a cycle, and two clusters joined by a bridge) the path through the rows is within 1% of the one optimal leaf ordering finds with a dendrogram (and shorter on two of them), against 6-9% longer for the dendrogram's own order and 18-25% (177% on the cycle) for a plain Fiedler vector. Up to clust_max rows are ordered exactly (5000 rows of 20 columns in about 0.4 seconds); larger data from clust_max landmarks with every other row placed beside its nearest one (200,000 rows in about 4 seconds).tree_cut_x/tree_cut_y = "auto" (and heat_map_clust(cut = "auto")) choose the number of clusters from the data. The dendrogram is cut into 2 to 20 clusters and the cut with the largest mean silhouette width kept, unless none reaches 0.25 (Kaufman and Rousseeuw's threshold for "no substantial structure"), when the dendrogram isn't cut. The silhouettes of all the cuts are computed together in compiled code, updating each cluster's distance sums from the smaller half of every split, so trying 19 cuts takes little longer than one; trees with more than clust_max leaves are scored on a sample of clust_max leaves (fixed seed; 0.3 seconds for 200,000 rows). The scores are returned in the tree's auto_cut element and heat_map_describe() reports the choice.clust_max (new in heat_map(), 5000 by default; max in heat_map_clust()) are clustered in two stages instead of exactly (see Breaking changes). A sample of clust_max rows, drawn with a fixed seed so that the result is the same each time, is clustered exactly and its dendrogram is split into groups of about 200 rows; every row joins the group with the nearest medoid, and each group is clustered on its own beneath the sample's dendrogram. Memory no longer grows with the square of the number of rows: on two cores, 100,000 rows of 30 values are clustered in about 0.7 seconds and a million in about 16 (exact clustering of 10,000 rows takes 1.5 seconds and 380 MB). On data with clear clusters, cutting the two-stage dendrogram gives the same clusters as the exact one. The dendrogram has a two_stage element and heat_map_describe() says when it was used. Branch support is skipped, with a warning, above clust_max.tree_cut_x/tree_cut_y assigns every column/row to a group (e.g. treatments, cell types) and a dendrogram is requested, each group is clustered on its own and the groups are joined at the top of the dendrogram, in their factor level order. Previously the dendrogram was dropped unless the groups happened to be contiguous in it. Clustering within groups also skips the clustering of all rows, and the groups share clust_max in proportion to their size (at least 50 rows each), so it is no slower than clustering all the rows at once (12 groups of 5,000 rows with optimal leaf ordering take 0.9 seconds). Non-numeric columns are shown after the clustered columns as a cluster of their own, as without groups.cell_uncertainty (a matrix of standard errors, interval widths or p-values) colours cells with a value-suppressing palette - the more uncertain a value, the fewer colours its part of the scale has and the closer they are to grey (8, 4, 2 then 1 colour with the default 4 levels; cell_uncertainty_breaks sets the thresholds), so a noisy estimate can't look like a strong one. The colour legend becomes a key with one row per uncertainty level.legend_col_density = TRUE traces the distribution of the values (a kernel density) over the colour legend - beside or below the heatmap - so you can see which colours most cells take and how much of the data sits near the ends of the scale, e.g. with robust limits.tree_support_x/tree_support_y (TRUE or a number of resamples) draw each dendrogram branch by its bootstrap support - the other dimension is resampled, the leaves are clustered again in the same way (within groups too) and each branch scores the proportion of resamples in which its set of leaves is a branch again. Branches with 90% support or more are thick black lines, 70-89% grey and lower support dashed, larger branches are labelled with their support (tree_support_labels) and the legend gains a key. The resamples use a fixed seed without touching the random number generator, so the support is the same every time; heat_map_info() returns it.cell_values takes a matrix holding the values behind each cell (e.g. heat_map_summarise(fun = "values"), new) and draws their distribution inside the cell - a density with a tick at the median, a histogram or a box plot (cell_glyph) - on a range shared by each column, row or all cells (cell_glyph_range). A heatmap of cluster medians then also shows the spread and shape behind each median, such as two populations hiding behind a middling one.bar_type_x/bar_type_y draw "box" plots or "violin"s of the distribution of several values for each column/row (as superheat's yr.plot.type or ComplexHeatmap's anno_boxplot()), e.g. marker distributions beside a heatmap of cluster medians. bar_values_x/bar_values_y take a list with a numeric vector per column/row, a matrix or data.frame with one column/row per column/row of x, or TRUE for the values of each column/row of the heatmap. Box plots are the default for distributions; bars are unchanged.tree_kmeans_y = 4 (or tree_kmeans_x) splits the rows (columns) into k-means clusters, a quick alternative to cutting a dendrogram of thousands of rows (as ComplexHeatmap's row_km). The clusters are computed on the scaled data from deterministic starts (equal slices along the first principal component, and k-means++ starts from fixed seeds), so they don't depend on the random seed, and are ordered so that similar clusters are neighbours. They then act like groups supplied to tree_cut_y: separated by splits, labelled by tree_label_y, and with tree_y = TRUE clustered within each cluster. heat_map_info()$row_clusters returns them.heat_map_summarise() reduces groups of rows (group_y) or columns (group_x) to one summary each - the mean, median, sum, minimum, maximum, standard deviation, count, proportion "detected" above a threshold, or any function - for pseudobulk heatmaps of single-cell data (as scater's plotGroupedHeatmap()) and dot plots that show the mean as colour and the proportion detected as cell size.axis_text_angle_x accepts an angle in degrees (e.g. axis_text_angle_x = 45) as well as the las codes 0 to 3. Rotated labels end at their ticks and slant towards the y axis text, so that long labels extend beneath (or above) it rather than into the dendrograms and legends. They are shrunk (or thinned) only when neighbouring labels would overlap across their baselines, which leaves them larger than perpendicular labels need on crowded heatmaps, and the layout makes room for the text and its overhang.heat_map_arrange() redraws heatmaps (from heat_map(), heat_map_dot(), heat_map_cor() or heat_map_align()) in one layout on shared colour limits and cell size limits - from the lowest to the highest limit of them all, centred when they are all centred on one value, and for aligned heatmaps block by block - so that panels drawn separately, e.g. one per condition, can be compared by eye. Dendrograms and clusters are reused.heat_map_check_colours() simulates the colours of a heatmap with deuteranopia, protanopia and tritanopia (the model of Machado et al. 2009, as colorspace simulates them) and reports the levels of annotations, of non-numeric cells, of stacked bars and of branch colours that become hard to tell apart (CIELAB Delta E below 10), and colour scales whose values half the scale apart look alike. options(HeatmapR.check_colours = TRUE) checks every heatmap as it is drawn and gives a message. heat_map_info() returns the categorical colours (keys) and colour scales (scales) it checks.heat_map_style() keeps settings of heat_map() (colour scales, text sizes, borders, legend options and the like) to reuse: heat_map(style = ) fills in the arguments that aren't given, and options(HeatmapR.style = ) gives every heatmap of the session the style, including those of heat_map_dot(), heat_map_cor(), heat_map_align() and heat_map_composition(). Arguments given win over the style, styles build on styles, names that aren't arguments of heat_map() are an error with the closest arguments suggested, and redrawn heatmaps keep the style they were drawn with.heat_map_app() opens an app (with the shiny and bslib packages) to make a heatmap without writing code. Its start page takes a CSV, TSV or text table, an Excel file (with readxl), an R data file or an AnnData file dropped on it, or an example (tables and single-cell data), and the app also starts from data in R. Each column of a table is given a role - shown, grouping the rows or annotating them - and single-cell data choose their features, groups and annotations; the heatmap can be a dot plot of groups or the correlations of the columns. Every argument of heat_map() has a setting, in sections with the most used first (plain labels and inputs made for them: segmented choices, steppers for cluster counts, palettes with swatches, colour limits over the distribution of the values, the colour-vision check), the argument's name and documentation beside each, and a search box. The heatmap is drawn again as settings change, reusing the clustering when only its look changes; hovering shows the values of its cells, and rows and columns selected on it can be shown alone, labelled or copied. Changed settings are listed with buttons to undo each, changes can be undone and redone, a setting that can't be drawn is flagged while the last heatmap stays, and settings of the last data's rows and clusters are reset when the data changes. The R code that draws the heatmap again (with only the changed settings) is written beside it, and the heatmap downloads as a PNG, PDF, SVG, TIFF or JPEG image at a chosen size, such as a journal column or a slide. Dot plots and correlations hide the settings of heatmaps they don't use (such as scaling, sampling the groups and annotating the cells, which would apply to the summary of the groups or the correlation matrix), and their own settings are kept while another plot is shown. Splitting grouped rows of a heatmap into clusters replaces the groups (whose columns then annotate the rows), and grouping the rows again drops the clusters. Single-cell data from files or R start grouped by their cell types or clusters, dots are split by columns with two values, and a selection on a dot plot keeps its groups. Settings that take R values are R code run with the data as x (e.g. bar_values_y = x$mpg); allow_code = FALSE limits them to plain values for apps hosted for others. The article Building heatmaps in the browser shows how the app works.heat_map_html() writes a heatmap (from heat_map(), heat_map_dot(), heat_map_cor(), heat_map_align() or heat_map_composition(), or data with the arguments of heat_map()) to a self-contained web page - one HTML file with the heatmap as an SVG (or PNG) image and its values, without dependencies or anything loaded from the web. Hovering over a cell shows its row, column and value (before and after scaling, its size, clusters and annotations); clicking a cell or a label selects its row or column, and dragging a box across cells, labels or dendrograms selects every row and column in it (Alt unselects), e.g. the genes of a cluster; the selection is listed, to copy one name per line or download with its values as CSV; a search box finds rows and columns by name; and the heatmap zooms. On touch screens a tap shows a cell's values and selects its row, and pressing and holding, then dragging, selects a box. Pages open in the RStudio or Positron viewer or a browser, and are embedded in HTML documents from R Markdown and Quarto (knit_print()).ggplot2: new heat_map_grob() turns a heatmap into a grob that redraws itself at the size it is given, so heatmaps sit beside ggplot2 plots in patchwork (p + heat_map_grob(hm)), cowplot or gridExtra layouts (needs gridGraphics). New heat_map_data() returns the cells, rows, columns and dendrogram segments of a heatmap as tidy data.frames, in display order and with the cell centres in the heatmap's plot co-ordinates, for rebuilding heatmaps with ggplot2, exporting what is shown, or drawing on the heatmap with base graphics. heat_map_info()$centers gives the centre of each row and column.heat_map_describe() writes a plain-language description of a heatmap - its size and scaling, what the colours show (naming the colours of the scale), the most extreme values, how the rows and columns are clustered and where each cluster is highest and lowest, branch support, uncertainty, split cells, annotations, bars, labelled rows and zoom panels - as alternative text for screen readers (fig.alt in R Markdown and Quarto) or a starting point for a figure legend.cell_split takes a second matrix of values, e.g. a stimulated condition beside the control in x, and splits each cell along its diagonal - the value in x fills the upper left half and the value in cell_split the lower right half, on one colour scale whose limits cover both (with scale, each column or row is scaled together with its counterpart). Two conditions are compared cell by cell without doubling the size of the heatmap; cell_split_labels names the halves in a legend key, and heat_map_info()$split_colours returns the colours of the lower halves.zoom_y (or zoom_x) enlarges a band of rows (columns) of a large heatmap in a panel beside it, so a few rows can be read together with the whole. The band runs from the first to the last of the chosen rows in the displayed order (names, indices or a logical vector; the first and last rows are enough), is outlined on the heatmap and joined to the panel by a wedge, and the panel keeps the cells' shapes, text, distributions and cluster gaps. A list enlarges several bands, each in its own panel (e.g. zoom_y = list(c("gene1", "gene9"), 40:55)); panels on the same side keep the order of their bands, are spread out so that they don't overlap and enlarge their rows by the same amount. zoom_side_y ("left" or "right") and zoom_side_x ("bottom" or "top") choose the side of each panel, one for all or in turn for the bands (c("left", "right") alternates). On the axis text side (the default) the panels replace the axis text with the names of their rows; on the other side they are drawn beyond the dendrogram and other tracks, which a lane carries each band across, are labelled on their outer side, and the heatmap keeps its axis text (or mark_y labels). zoom_size_y sets the width of the panels relative to the heatmap (by default as wide as the heatmap for panels as long as it, and narrower, down to 0.3, for shorter panels, so that enlarging a few rows - e.g. one cell of single-cell data - doesn't halve the heatmap) and enlarged cells are at most 0.3 inches along the band, but never smaller than the cells of the heatmap; gaps between clusters keep their size in the panels. heat_map_info() returns the enlarged rows and columns (a list for several bands).heat_map_cor() draws the correlations between the columns of a data set (or a correlation matrix) with one order for the rows and columns, so the diagonal runs from the top left to the bottom right. The variables are clustered on one minus their correlations, and by default the lower triangle shows ellipses, the upper triangle the coefficients and the diagonal the names of the variables, as in corrplot's mixed layout; lower, upper and diag take any cell shape. Kendall correlations are counted in compiled code by Knight's O(n log n) algorithm, as Kendall distances are: 20,000 cells and 12 markers take 0.1 seconds instead of six and a half minutes with cor(), with cor()'s results for each use.cell_shape = "ellipse" shows values between -1 and 1, such as correlations, by the shape of an ellipse (the outline of a bivariate normal distribution with that correlation, rising for positive and falling for negative values, narrower the stronger the correlation); "number" writes the value (or cell_text) of each cell in the colour of the cell, darkened where it would be too light to read on white; and "none" leaves a cell empty apart from its text. A matrix of shapes mixes them, e.g. ellipses below the diagonal and numbers above it. cell_grid_line_col draws a light grid around the cells that aren't empty, which keeps the rows and columns visible around shapes that don't fill their cells. heat_map_info()$shapes and the shape column of heat_map_data() give the shape of each cell.mark_y and mark_x (names, positions as displayed or a logical vector), e.g. a few marker genes on a heatmap of thousands. The labels replace the axis text, are spread out so that they don't overlap and are connected to their rows by leader lines; mark_labels_x/mark_labels_y set the text and mark_line_col the line colour.cell_raster, "auto" by default: heatmaps with more than 50,000 cells of full-size rectangles). A 1000 x 1000 heatmap draws in about 0.15 seconds and its PDF is a fraction of the size, with the same colours as vector cells; cell borders are drawn as grid lines. heat_map_info()$raster reports whether raster images were used.heat_map_info()$layout gives the bounding box of every component of the heatmap (axis text, dendrograms, bars, legends, titles) in inches, so layouts can be checked programmatically.heat_map_info() extracts the data, row/column order, cluster assignments, dendrograms, cell colours/sizes and scale limits from a heatmap.cell_col_scale and cell_col_palette accept palette names from hcl.pals() (e.g. cell_col_scale = "viridis"), cell_col_scale also takes palette functions such as colorRampPalette(c("blue", "white", "red")), and cell_col_scale_center centres the colour scale on a value for diverging palettes (e.g. z-scores centred at 0).cell_size_scale_method = "area" scales cell area rather than width with value, and cell_size_scale_range sets the smallest/largest relative cell sizes so that the smallest values remain visible.legend = c("both", "levels").cell_col_alpha accepts a matrix (or a vector recycled down the columns of x) to set the transparency of each cell.group_by takes several metadata columns (or a data.frame of variables) in heat_map(), heat_map_dot() and heat_map_summarise(), grouping the cells by their combinations, e.g. group_by = c("cell_type", "condition") for each cell type in each condition (scanpy's multi-column groupby, Seurat's split.by). Groups are ordered by the first variable, then the next within it, and named "CD4 T / control". heat_map() splits and labels the cells of each combination (group_max takes that many cells from each), heat_map_dot() gives one row per combination, grouped by the first variable in labelled strips and labelled by the others, and heat_map_summarise() one row per combination (e.g. a pseudobulk of each cell type in each sample with fun = "sum").heat_map_dot(split_by = "condition") splits each dot down the middle by a variable with two values - the cells of the group with the first value in the left half and those with the second in the right half, each half coloured by its mean and sized by its proportion detected on the same scales (the means are scaled within each feature across the groups and both values), so two conditions are compared in each dot without doubling the rows. In heat_map(), cell_split now also splits circles (cell_shape = "circle") down the middle, cell_split_size sizes their right halves on the size scale of cell_size (with cell_size = TRUE, by the values of cell_split), halves with a missing size are not drawn, and heat_map_info()$split_sizes returns the sizes of the right halves.x (axis_text_y, axis_text_x) labels each row (column) by name, wherever clustering draws it.heat_map() and heat_map_summarise() read single-cell data: SingleCellExperiment and SummarizedExperiment objects, Seurat objects, flowFrame and flowSet objects and sparse matrices (dgCMatrix and the other Matrix classes), as well as matrices and data.frames. Cells are drawn as rows and features as columns. The packages are only needed to read their own objects (SummarizedExperiment, SeuratObject and Matrix are suggested; flowFrames and flowSets are read without flowCore).features (genes, proteins or markers by name - row names, gene symbols in rowData(), flow marker or channel names - or index; missing features are an error that names them), assay (an assay of a SingleCellExperiment, by default "logcounts", else "exprs"; an assay or layer of a Seurat object, by default "data"), group_by (a metadata column or a vector, grouping the cells as tree_cut_y does, labelled by group) and annotation_by (metadata columns drawn as annotation tracks). swap_axes = TRUE draws features as rows and cells as columns, as Seurat's DoHeatmap() does.heat_map_downsample(groups, n = 500) keeps at most n cells of each group, drawn with a fixed seed, and heat_map(..., group_max = n) uses it so that small clusters stay visible next to large ones. Arguments with one value per cell are kept for the cells shown.heat_map_summarise() finds the mean, sum, number, proportion detected and median of each feature in each group from the non-zero values, in compiled code (all 20,000 genes of 50,000 cells in 20 groups: 0.2-0.6 seconds for means, 0.6-1.7 seconds for medians), giving the same numbers as for the dense data. A message warns when dense values would take more than 1 GB.plot(hm, data = sce2)) read it with the same single-cell arguments.heat_map_dot() draws dot plots of groups of cells, one row per group and one column per feature: the colour of each dot is the group's mean scaled within the feature (scale = "zscore", clipped to scale_limits = c(-2.5, 2.5), as Seurat's DotPlot(); "range" for 0 to 1, as scanpy's standard_scale = "var"; or "none") and its area the proportion of the group's cells detected (above threshold), with a size legend in percent. dot_min leaves out dots below a proportion, swap_axes = TRUE draws features as rows, and every heat_map() option (e.g. tree_y = TRUE for a dendrogram of the groups, or bar_values_y = "n" for the number of cells in each group) can be added.heat_map_markers() ranks the marker features of each group by their AUC against all other cells (the Wilcoxon rank-sum statistic over the numbers of cells, ties counting half, as presto's wilcoxauc()), keeping features detected in at least min_detected (0.1) of the group's cells with a mean at least min_logfc (0.25) above the other cells'. Each feature marks the group where its AUC is highest, and the top n (5) of each group are marked in the top column. Each AUC has the two-sided p-value of the Wilcoxon rank-sum test (p_value, as wilcox.test(exact = FALSE) finds it, with corrections for ties and continuity) and its false discovery rate within the group (p_adj, Benjamini-Hochberg over every feature scored). The AUC is computed in compiled code from one ranking of each feature, with the zeros of sparse data sharing a rank, only for features that pass the filters (all 20,000 genes of 50,000 cells in 1.4 seconds on two cores; 30 markers of a million cells in 0.6 seconds).heat_map_dot() takes transform = "asinh" and cofactor, as heat_map_summarise() does: the values of the cells are transformed before their means are taken and threshold applies to the transformed values, so cytometry intensities need no transform beforehand (the zeros of sparse data stay zeros, without a dense copy).heat_map_score() scores each cell for each of several sets of genes (e.g. pathways or programs) - the mean of the set's genes z-scored across the cells (the default), their plain mean, or a module score (method = "module", as Tirosh et al. 2016 and Seurat's AddModuleScore(): the mean less that of control genes drawn from the same bins of mean expression, with a fixed seed). The scores are computed in compiled code from the non-zero values of sparse data and returned as cells by sets with the cells' metadata, so heat_map(), heat_map_dot() and heat_map_summarise() draw and summarise them by group_by. The markers of heat_map_markers() give one set per group; genes that aren't in the data are left out with a message.heat_map_composition() counts the cells of each group (group_by, e.g. cell type) in each sample (sample_by) and draws the share of each sample's cells in each group (or percentages, counts or centred log-ratios, values = "clr", the transform of Aitchison 1982 for comparing compositions), one row per sample, with the samples clustered, annotated by their metadata (annotation_by, e.g. condition) and their numbers of cells as bars. Only the cells' metadata is read, so any single-cell data or a data.frame of metadata can be given.heat_map_contrast() compares two conditions (condition_by) in each group of cells (group_by) with the samples (sample_by, e.g. donors) as replicates, and draws the log fold change of each feature in each group on a diverging scale with stars for adjusted p-values (or returns the table, draw = FALSE). The cells of each group in each sample are summarised - their mean (of log-normalised or, with transform = "asinh", cytometry values) or the log2 counts per million of their summed counts (values = "logcpm", leaving out features with fewer than min_counts counts as edgeR's filterByExpr() does) - and the samples of the two conditions with at least min_cells cells are compared by moderated t-tests (Smyth 2004; the same statistics as limma's lmFit() and eBayes(), compared as a black box), so the thousands of cells of a sample don't count as independent replicates (Squair et al. 2021). P-values are adjusted within each group (Benjamini and Hochberg 1995).heat_map_align() draws several data sets measured on the same rows - e.g. the genes (RNA) and surface proteins (ADT) of the same groups of cells, or the expression and module scores of the same cell types - as one heatmap, as ComplexHeatmap's heatmap lists do. The rows are matched by name and lined up in one order; each data set is a block of columns, labelled by its name, scaled on its own (scale) and drawn on its own colour scale with its own legend (cell_col_scale, cell_col_scale_limits and cell_col_scale_center take one setting for all or a list with one per data set). The rows are clustered (tree = TRUE) or split by k-means (tree_kmeans) on the main data set alone, and axis = "y" stacks data sets that share their columns as blocks of rows. The blocks are drawn by heat_map(cell_col_blocks_x = ) (or cell_col_blocks_y), which gives any heatmap columns (rows) on their own colour scales; heat_map_info()$col_blocks returns each block's limits and heat_map_describe() its range and extremes.DelayedArray objects such as the HDF5Matrix and TENxMatrix of HDF5Array, and BPCells matrices such as those of Seurat v5 layers - are read a block of features at a time. Each block of every cell is realised in memory as a dgCMatrix (sparse data) or a dense matrix and summarised by the compiled code used for data in memory, so heat_map_summarise(), heat_map_markers(), heat_map_score(), heat_map_dot() and heat_map_smooth() of every gene of a large data set hold one block in memory at a time (2^24 values of dense data, eight times more of sparse data; options(HeatmapR.block_values = ) sets it) and give the same results as the data in memory. Heatmaps read only the chosen features of the cells shown, as before. A DelayedArray or BPCells matrix given on its own is read like a sparse matrix, one cell per row. A DelayedArray that only wraps a matrix or sparse matrix in memory, without delayed operations, is read as that matrix.heat_map_read_h5ad() reads .h5ad files written by scanpy and anndata (or downloaded from CELLxGENE) without Python, reticulate or a conversion, using hdf5r. The metadata of the cells (obs) and features (var) are read at once - categoricals as factors, nullable columns with their missing values, from anndata 0.7 to 0.13 - and X, a layer (assay = "counts") or raw is read the first time it is used, as a dgCMatrix (CSR and CSC matrices without a copy) or a dense matrix, and kept for the next heatmap. Every single-cell function reads the object, or an .h5ad file name, with group_by naming columns of obs and features named by the index of var or a column of symbols or IDs (e.g. CELLxGENE's feature_name). AnnData objects of the anndata R package are read too.heat_map_dot(markers) draws the markers returned by heat_map_markers() as a dot plot, one row per group and one column per top marker (grouped by the group it marks, or chosen with features), coloured by the log fold change on a diverging scale centred at 0 and sized by -log10 of the adjusted p-value (as scanpy's rank_genes_groups_dotplot()). marker_colour and marker_size choose other columns ("auc", "p_value", "mean_in", "detected_in", ...); a feature has a dot in each group where it passed the filters of heat_map_markers(), in all of them with min_detected = 0 and min_logfc = -Inf.features takes a named list of features (e.g. list(T = c("CD3E", "CD3D"), B = "MS4A1")) or the markers from heat_map_markers(), grouping the columns of heat_map() and heat_map_dot() by group with labels, as scanpy's var_group_positions brackets do.legend_size_format labels the size legend: "percent" for proportions, or a function.heat_map(bimodal = TRUE) and heat_map_dot(bimodal = TRUE) outline, with a thick border, the cells of a summary in which a group's cells look like two populations, so that clusters that should perhaps be split can be looked into. heat_map_summarise(bimodal = TRUE) keeps the dip statistic of Hartigan and Hartigan (1985) of each feature in each group (fun = "dip" gives the dips themselves), found in compiled code so that the measurement and its transformation don't make peaks of their own: only from the values above threshold (leaving out piles of zeros and the lower of the two mirrored peaks the asinh transform with a small cofactor makes of a negative population), after the transform and before scaling, with discrete values such as counts spread over the gaps between them, and without the piles at the ends of continuous values (clipping, saturated detectors). A cell is outlined when its dip is at least 0.01 (or the number given to bimodal) and above the dip one population of as many cells exceeds 1% of the time. Populations that differ in several features at once, such as CD4 and CD8 T cells in one cluster, each without the other's markers, are found from the dip of the cells' scores on the first principal component of the features' correlations within the cluster, when it carries co-variation, and outlined in the features that drive it. On simulated data no single population was outlined - negative populations under a small cofactor, integer and low counts, skewed and saturated ones - while two populations 3 standard deviations apart were in groups of 5,000 cells. bar_values_y = "dip" draws the largest dip of each row, heat_map_info()$bimodal returns the dips and heat_map_describe() names the cells outlined. The dip is HeatmapR's own implementation of Hartigan's algorithm (AS 217), without new dependencies.heat_map_example() simulates the data of the examples and guides: blood cells of six types with their marker genes ("rna"), cytometry events in clusters of samples in two conditions ("cytometry") and cells of a branching trajectory ("trajectory"), as a data.frame of cells or a SingleCellExperiment, the same each time and without changing the session's random numbers.flowFrame) are found in compiled code, without copying the data (a million cells of 30 markers in 0.4 seconds).heat_map_scale(method = "asinh", cofactor = 5), heat_map(transform = "asinh", cofactor = ) and heat_map_summarise(transform = "asinh", cofactor = ) take the inverse hyperbolic sine of each value divided by the cofactor (5 for mass cytometry, about 150 for flow cytometry; one, or one per marker). Sparse data stay sparse.heat_map_scale(method = "quantile", q = 0.01) (heat_map(scale = "quantile", scale_q = )) maps the q and 1 - q quantiles of each column to 0 and 1 and clips the values beyond them, so that a few bright cells don't compress the colours of the rest. The quantiles are those of quantile(), and missing values stay missing.heat_map_summarise() transforms and scales: scale (a scaling method for each feature) is applied to the summaries (scale_when = "last", the default) or to the cells before summarising ("first"). group_by takes several columns, e.g. c("cluster", "sample") for the medians of each cluster in each sample, in rows named "cluster / sample".group_by keep the number of cells of each group (attribute "n"), their numbers in each part of stack_by (e.g. the condition; attribute "stack") and the metadata that is the same for every cell of a group (attribute "groups", e.g. the condition of each sample). heat_map() reads them: bar_values_y = "n" draws the number of cells of each group, and heat_map(s, group_by = "cluster", annotation_by = c("sample", "condition")) groups and annotates the rows of a summary by its groups.bar_type_x/bar_type_y = "stack" draws the parts of each column or row (a matrix with one column or row per part, or "n" for the counts of stack_by) as a stacked bar, and "fill" their proportions, with one colour per part (bar_fill_*, named by part) and a legend of the parts.median() finds them: a million cells of 30 markers in 20 clusters in 0.3 seconds, or 1.3 seconds with the asinh transform and quantile scaling.heat_map_smooth() smooths features along a trajectory: the cells are binned along along (e.g. a pseudotime column; bins = 100 bins with equal numbers of cells, or of equal width with binning = "width") and each feature's mean found over the span bins around each bin (5 by default), weighted by their cells, in compiled code (2,000 genes of 50,000 cells in 0.3 seconds, all 20,000 in 1.1). Branching trajectories are smoothed one lineage at a time, from several along columns (one per lineage) or group_by. The result has one row per bin and one column per feature, with the cells of each bin (attribute "n") and its lineage and position (attribute "groups") for heat_map(group_by = "lineage", annotation_by = "pseudotime").tree_order_x/tree_order_y = "peak" (and heat_map_clust(order = "peak")) order features by where they are highest (ties by where they are lowest), so that cascades along a trajectory run diagonally. It needs no dendrogram, orders a dendrogram's branches by the mean peak of their leaves, and keeps within groups of features; the bins of heat_map_smooth() are placed by their position along the trajectory, so the peaks of lineages are compared by when they occur. heat_map_clust(order = ) also takes a numeric vector, ordering branches by the mean value of their leaves.stats::dist(), one minus Pearson, Spearman or Kendall correlations and cosine distances, with missing values), every linkage, cutting dendrograms, two-stage clustering, k-means, optimal leaf ordering, branch support and the dendrogram computations; only a distance function supplied by the user runs in R. Clustering 5,000 rows takes 0.3 seconds on two cores instead of 2.0 in HeatmapR 1.1.0, and two-stage clustering of 100,000 rows about 0.7 seconds (HeatmapR 1.1.0 would need the 40 GB distance matrix of every pair of rows).stats::dist(), cor(), stats::hclust() and stats::cutree() everywhere: Kendall distances between 1,000 rows take 0.05 seconds instead of 4.3 with cor() (Knight's O(n log n) algorithm per pair, or cross-products of sign vectors), Spearman distances with missing values 1.2 seconds for 3,000 rows instead of two minutes (computed pair by pair, giving cor()'s results), binary distances (as bit vectors) 12 times faster and Canberra distances 3 times faster, centroid and median linkage of 5,000 rows take 0.3 seconds instead of 1.0 with stats::hclust(), single linkage needs no matrix of distances (5,000 rows in 0.1 seconds instead of 0.27), and cutting a dendrogram of a million leaves into clusters takes 0.1 seconds instead of five minutes with stats::cutree(), whose time grows with the square of the number of leaves.tree_kmeans_x, tree_kmeans_y) use a compiled Hartigan-Wong algorithm (Hartigan and Wong 1979), which gives stats::kmeans()'s clusters without its copies of the data. It runs from the centres of equal slices of the rows along their first principal component (computed from the covariance matrix in one pass) and from nine greedy k-means++ starts (Arthur and Vassilvitskii 2007; Grunau et al. 2023; HeatmapR's own random numbers from fixed seeds), fitted on separate threads, and keeps the clusters with the smallest within-cluster sum of squares; for more than 20,000 rows the starts are compared on a sample of 20,000, the distinct fits of the sample within 2% of the best (at most three) are compared on every row by the sum of squared distances to their closest centres, and every row is then fitted once, from the centres of the best. On clusters of very different sizes, which slices split badly, the within-cluster sum of squares was 19-31% smaller on three simulated examples, and nothing changes where the slices do best. On 50,000 cytometry events (raw, asinh and z-scored, 4-8 clusters, four subsamples) the within-cluster sum of squares was at most 0.6% above that of stats::kmeans(nstart = 10) and 2.7% below it on average, in about 0.1 seconds against 2-3. Distances to the centres are computed four points or centres at a time (AVX2), bounds on each point's distances skip most of the checks of the quick-transfer stage, the k-means++ starts are drawn in blocks of rows on several threads (the same starts whatever the number of threads), and rows are compared by their numbers rather than by duplicated()'s 15-digit strings when a message needs the number of distinct rows: on two cores, 1,000,000 rows of 30 random values take 3.0 seconds for 10 clusters, and 200,000 rows 2.3 seconds for 30 clusters. heat_map_scale() scales in compiled code (1,000,000 x 30 in 0.7 seconds instead of 5.5, with identical results).std::threads, so they run in parallel on macOS too (Apple's compiler has no OpenMP), and results are the same whatever their number. New heat_map_threads() sets the number of threads (also options(HeatmapR.threads = )): by default one per physical core, within the CPUs the process may use, OMP_THREAD_LIMIT and OMP_NUM_THREADS when they are set, at most two under R CMD check's core limit, and one in processes forked from R (e.g. by parallel::mclapply()). Two threads take about half the time of one for two-stage clustering, branch support and optimal leaf ordering. The linkage stops sharing its scans when waiting for the other threads takes longer than the work they save (e.g. when another process keeps a core busy), so extra threads never make it slower.colorRamp(), rgb() and adjustcolor() compute them, so the colours are the same), dendrogram segments are computed directly from the merge matrix, and large heatmaps (more than 50,000 cells, cell_raster = "auto") are drawn as raster images passed to the device as packed colours (a nativeRaster), with cell borders as grid lines. A 2000 x 50 heatmap draws in 0.07 seconds instead of 19 in HeatmapR 1.1.0 (0.13 instead of 27 with a row dendrogram), a 1000 x 1000 heatmap in 0.15 seconds instead of 174 and a 50,000 x 30 heatmap in under 0.1 seconds (HeatmapR 1.1.0 takes several minutes); the PDF of a raster heatmap is a fraction of the size.print(), plot(), heat_map_grob() (which redraws whenever its space changes) and heat_map_save(x = ) - reuses its dendrograms, k-means clusters and branch support when the data and the arguments they depend on are unchanged, so a redraw only draws: a 20,000-row heatmap with optimal leaf ordering redraws in 0.4 seconds instead of 3. A redraw with a new cut, height scaling or leaf order reuses the clustering (and branch support, which depends only on the branches) and only redoes those steps.heat_map objects keep their cell colours as packed colours (4 bytes a cell, written as strings when heat_map_info() is called), and constant cell sizes and shapes as one value (heat_map_info() returns them as before), and no second copy of the data: a 50,000 x 30 heatmap saves to 18 MB.cell_col_palette.cell_border_line_col = NA or cells outside a cell_border_mask) and annotation tracks no longer show faint lines between neighbouring rows or columns. Anti-aliasing drew the shared edge of two touching rectangles partly transparent in both, so the background showed through as hairline seams at some sizes - in PNG files and in PDF and SVG viewers. Each opaque cell now reaches one device unit (a pixel, or a point in a PDF) into the cell to its right and the one above it, which are drawn after it and cover the overlap; cells along cluster gaps and the edges of the heatmap, transparent cells and cells with borders are drawn as before. Cells with masked borders are drawn after the others, so their borders are no longer half covered by their neighbours.scale = "row" in heat_map() or heat_map_scale()) returned transposed values, silently scrambling the data. Values are now scaled within each row as documented.cell_size matrix on a different scale to x could fill the plot with black shapes, and legend = "size" on its own drew nothing.heat_map(iris[1:10, 1:4], cell_size = TRUE)); it uses fewer labels or is omitted with a warning. When fewer than five sizes are shown, the legend is now centred in its space instead of being offset.axis_text_x/axis_text_y labels caused an error, and a fully missing axis_text_y used the number of columns instead of rows.bar_fill_x = c(B = "red", A = "blue")) were matched to the wrong bars unless already in column order.NaN, which previously rendered as empty cells.tree_x = FALSE/tree_y = FALSE now disable clustering, and a user supplied hclust object is no longer discarded when a single tree_cut_x/tree_cut_y is supplied. Supplying a cut with an hclust object that already contains clusters (e.g. from heat_map_clust(cut = 3)) now re-cuts the tree instead of keeping the old clusters.dist objects supplied to tree_x, tree_y or heat_map_clust(tree = ) were ignored; they are now clustered with clust_method as documented.tree = TRUE drew no dendrograms; it now draws both, like tree = "both".legend = "none" and legend = NA now remove the legend, rather than reserving empty space for it.cell_text = TRUE.heat_map_save() now accepts .jpg and .tif extensions and is case-insensitive (e.g. .PNG), and compresses TIFF files without loss (compression = "lzw", 20 times smaller for a 7 x 7 inch heatmap at 300 dpi) unless another compression is given.heat_map_cor() drew missing correlations (e.g. of a constant column) as empty white circles; their cells are now left blank in the grid.margins must be four numbers or NA (shorter vectors were padded with NA, longer ones cut), and settings of cluster labels (tree_label_col_x, tree_label_text_size_y, ...) with neither one value nor one per cluster are recycled with a warning (they were recycled silently, and rows from the bottom).dgCMatrix from a Seurat or SingleCellExperiment object) is read with cells as rows; when group_by has one value per column, or features are row names, the error now says so and suggests t(x).heat_map() fails while saving, the graphics device opened by heat_map_save() is closed and the saving state is reset, so subsequent plots are no longer drawn into the file.cell_shape now work as documented.heat_map_reset() now closes the graphics device opened by heat_map_save(); previously the saving state was cleared before the device could be closed.heat_map_scale() stored a single numeric data.frame column as a one-column matrix.png() or pdf()) returned an empty recorded plot because file devices don't keep a display list by default; heat_map() now enables recording on the drawing device.heat_map_scale() computes its statistics from the finite values: an infinite value turned every value of its row or column into NaN.dist_method = "cosine") with missing values compare each pair of rows on the columns both have (as dist() does for the other distances); they were NaN, which failed. Infinite values are left out of distances in the same way.tree_cut_* < 1) failed; it now gives one more cluster than the number of merges above the height.data-raw/logo.R draws it).fit_text = TRUE), and axis labels that still don't fit at half size are thinned to every second, third, ... label;margins that are too small for the axis text and label beside them shrink that text (to 40% at most) rather than drawing it over itself.tree_label_col_x/tree_label_col_y = "grey85"), at a contrast of 15:1; the dark grey strips of earlier versions had black text at 3.7:1, below the 4.5:1 that the Web Content Accessibility Guidelines ask of text. The text colour (tree_label_text_col_x/tree_label_text_col_y = "auto", the new default) is white on dark strips and black on light ones, whichever contrasts more with each strip's colour, so strips of any colour stay readable.cell_border_mask are always drawn.NA) axis labels no longer get a tick mark, so axis_text_y = NA removes the y axis text of a large heatmap instead of leaving a solid bar of ticks.ggdendro functions of 1.1.0 (dendrograms are drawn by HeatmapR's own code) and unused internal helpers.heat_map_info() and the recorded drawing calls) rather than only that no error occurs, and covers over 99% of the code.heat_map(), heat_map_clust(), heat_map_scale(), heat_map_summarise(), heat_map_markers(), heat_map_dot(), heat_map_cor(), heat_map_smooth() and heat_map_example(), the vignette and the articles cite the papers behind them. The DOI of Murtagh and Contreras (2012) in the Description is corrected.dist = "cosine" parameter in heat_map_clust() and heat_map().
tree_cut values between 0 and 1 are interpreted as absolute cosine distance thresholds rather than proportional tree heights. This allows for more intuitive clustering based on actual cosine distance values.tree_cut_x = c(3,4,5))tree_cut_x = c(1,1,1,2,2,2,2,3,3,3))cell_border_mask parameter to selectively apply colored borders to specific cells. Accepts a logical matrix of the same dimensions as the input data, where TRUE indicates a cell should have a colored border and FALSE indicates a transparent border.cell_col_scale_limits parameter to set custom limits for colour scales, enabling consistent colour scales across multiple heatmaps (e.g., cell_col_scale_limits = c(0, 100)).cell_size_scale_limits parameter to set custom limits for size scales when using cell_size = TRUE, ensuring consistent size scaling across multiple plots.