deepcinac.cinac_benchmarks

Module Contents

Classes

CinacBenchmarks

Used to plot benchmarks.

SessionForBenchmark

From a directory content including a yaml config file, will read ground truth + infered data

Functions

build_spike_nums_dur(spike_nums, peak_nums[, traces, ...])

Build a "raster dur", meaning a 2d binary array (n_cells * n_frames) representing when a cell is active

get_raster_dur_from_traces(traces[, ...])

Allow to build a raster dur based on all putative transient from the fluorescence traces.

compute_stats_over_gt(raster_gt, raster_to_evaluate, ...)

Compute the stats based on raster dur

load_data_from_np_or_mat_file(file_name[, attr_name])

Load data from a numpy or matlab file (.npz, .npz or .mat)

bin_raster(spike_nums)

Take a binary 2d array (n_cell*n_frames) and return a binned version,

get_raster_dur_spikes_and_traces(spike_nums, traces)

Take a binary raster (n_cells*n_frames) representing spikes or onsets of transient,

do_traces_smoothing(traces)

benchmark_neuronal_activity_inferences(inferences_dir, ...)

Evaluate the performances of different inferences methods. Every directory in the main

extract_age(label)

deepcinac.cinac_benchmarks.build_spike_nums_dur(spike_nums, peak_nums, traces=None, fluorescence_threshold=None)

Build a “raster dur”, meaning a 2d binary array (n_cells * n_frames) representing when a cell is active based on the onset and peak on transients. :param spike_nums: (2d array binary n_cells * n_frames), represent the onsets of cell activations (transients) :param peak_nums: (2d array binary n_cells * n_frames), represent the peaks of cell activations (transients) :param traces: (2d array float n_cells * n_frames) fluorescence traces of the cells :param fluorescence_threshold: if a peak amplitude is under the threshold, we don’t consider i

Returns:

deepcinac.cinac_benchmarks.get_raster_dur_from_traces(traces, fluorescence_threshold=None)

Allow to build a raster dur based on all putative transient from the fluorescence traces. :param traces:(2d array float, n_cells *n_frames) fluorescence traces :param fluorescence_threshold: None or otherwise 1xlen(traces) array with for each cell the threshold under which we should not take into account a peak and the transient associated. The value is without normalization. :return:

deepcinac.cinac_benchmarks.compute_stats_over_gt(raster_gt, raster_to_evaluate, traces, raster_predictions=None, fluorescence_threshold=None)

Compute the stats based on raster dur :param raster_gt: 1d binary array (n_frames or 2d binary array (n_cells, n_frames), 1 when the cell is active :param raster_to_evaluate: same sahpe as raster_t, binary, 1 when a cell is active :param traces: (1d or 2d array, same shape as raster_gt), should be smoothed, allows to detect :raster_predictions: same shape as raster_gt, but float array, probability that the cell is active at each frame. Can be None, if ground_truth is not based putative transients fluorescence_threshold: if not None, float value representing a low threshold for

# for transients used for benchmarks (possible transients), transients below the threshold are not considered

Returns

two dicts: first one with stats on frames, the other one with stats on transients

Frames dict has the following keys (as String): TP: True Positive FP: False Positive FN: False Negative TN: True Negative sensitivity or TPR: True Positive Rate or Recall specificity or TNR: True Negative Rate or Selectivity FPR: False Positive Rate or Fall-out FNR: False Negative Rate or Miss Rate ACC: accuracy Prevalence: sum positive conditions / total population (for frames only) PPV: Positive Predictive Value or Precision FDR: False Discovery Rate FOR: False Omission Rate NPV: Negative Predictive Value LR+: Positive Likelihood ratio LR-: Negative likelihood ratio

transients dict has just the following keys: TP FN sensitivity or TPR: True Positive Rate or Recall FNR: False Negative Rate or Miss Rate

class deepcinac.cinac_benchmarks.CinacBenchmarks(results_path, colors_boxplots=None, verbose=0)

Used to plot benchmarks. To do so 4 steps: Create an instance of CinacBenchmarks Add data using add_inference_to_benchmark() Add color for session if you want to using color_by_session() then call one the method to plot stats

color_by_session(session_id, color)

Attribute a cell to a color, allowing for example to give to cell of the same :param session_id: (str) :param color:

Returns:

add_ground_truth(session_id, ground_truth, smooth_traces)
Parameters

  • session_id –

  • ground_truth –

  • smooth_traces –

Returns:

add_inference_to_benchmark(session_id, inference_to_benchmark_id, raster_to_evaluate, cells_to_benchmark, raster_predictions=None)
Parameters

  • session_id –

  • inference_to_benchmark_id –

  • raster_to_evaluate –

  • raster_predictions –

  • cells_to_benchmark –

Returns:

evaluate_metrics()

Should be called after all inference to benchmark have been added and before generating the plots Returns:

_evaluate_metrics_on_a_cell(session_id, inference_to_benchmark_id, cell)
Parameters

  • session_id – session from which the activity is recorded

  • inference_to_benchmark_id – (str) id of the inference to benchmark

  • cell – int

Returns: a dict with same keys as inferences_to_benchmarks and values…

plot_boxplot_predictions_stat_by_metrics(description, time_str=None, colorfull=True, white_background=True, for_frames=False, save_formats='pdf', dpi=500)

PLot the boxplot regarding the predictions done by cinac classifiers. :param description: (str) will be added to file_name :param time_str: (str) timestamps for file_name, optional :param save_formats: str or list of str (.pdf, .png etc…) :param dpi:

Returns:

plot_boxplots_full_stat(description, time_str=None, for_frames=True, with_cells=False, color_cell_as_boxplot=False, box_plots_labels=None, colorfull=True, white_background=True, stats_to_show=('sensitivity', 'specificity', 'PPV', 'NPV'), title_correspondance=None, alpha_scatter=1.0, with_cell_number=True, put_metric_as_y_axis_label=False, using_patch_for_legend=False, with_legend=False, save_formats='pdf', dpi=500)
Parameters

  • description –

  • time_str –

  • for_frames –

;param box_plots_labels: if not NOne, list of str, allowing to choose the order of the boxplots. The labels should be existing one, otherwise it will crash :param with_cells: if True, display a scatter for each cell :param save_formats: :param title_correspondance: dict or None, keys are the stats to show, and the value is another string that will be displayed in the title :return:

plot_boxplots_for_transients_stat(description, colorfull=True, time_str=None, save_formats='pdf')
stats_on_performance(stats_to_evaluate, data_labels=None, title_correspondance=None, for_frames=False)

Compare with a wilcoxon test the performance over all pair of labels (methods) :param stats_to_evaluate: list of str :param title_correspondance: dict

Returns:

plot_boxplots_f1_score(description, colorfull=False, time_str=None, for_frames=True, with_cells=False, color_cell_as_boxplot=False, box_plots_labels=None, white_background=False, with_legend=False, using_patch_for_legend=True, alpha_scatter=1, with_cell_number=True, put_metric_as_y_axis_label=False, save_formats='pdf', dpi=500)
Parameters

  • description –

  • time_str –

  • (bool) (colorfull) – if True, the boxplots are filled with color, one different for each

(based on brewer colors) :param white_background: (bool) if True white background, else black background :param for_frames: :param with_cells: if True, display a scatter for each cell :param save_formats: :return:

plot_boxplots_proportion_frames_in_transients(description, time_str=None, colorfull=True, white_background=False, only_this_key=None, with_scatter=True, with_cell_text=True, alpha_scatter=0.6, using_patch_for_legend=False, with_legend=False, put_metric_as_y_axis_label=True, save_formats='pdf', dpi=500)
Parameters

  • description –

  • time_str –

param only_this_key: (str): key of the label, if not None, then only the boxplot corresponding will be displayed :param save_formats: :return:

deepcinac.cinac_benchmarks.load_data_from_np_or_mat_file(file_name, attr_name=None)

Load data from a numpy or matlab file (.npz, .npz or .mat) :param file_name: :param data_descr: string used to display error message if the file is not in the good format :param attr_name:

Returns:

deepcinac.cinac_benchmarks.bin_raster(spike_nums)

Take a binary 2d array (n_cell*n_frames) and return a binned version, with output shape 2d array (n_cell*n_frames//2) :param spike_nums:

Returns:

deepcinac.cinac_benchmarks.get_raster_dur_spikes_and_traces(spike_nums, traces)

Take a binary raster (n_cells*n_frames) representing spikes or onsets of transient, and based on the smoothed traces, detect putative transients and extend the spikes/onsets so that the duration of the rise time of the transient in which they are fully ‘active’ :param spike_nums: :param traces:

Returns:

class deepcinac.cinac_benchmarks.SessionForBenchmark(dir_to_explore, inferences_to_benchmark, predictions_keywords_dict, default_predictions_threshold)

From a directory content including a yaml config file, will read ground truth + infered data

get_inference_ids()
extract_data_from_cinac_file(cinac_file, cells_to_add, for_ground_truth, label=None)

From a cinac_file, :param session_id: str :param cinac_file: :param cells_to_add: :param for_ground_truth: (bool) to set ground truth or add some inference :param label: (str), necessary if for_ground_truth is False, allow to identify the inference

Returns:

add_it_to_cinac_benchmark(cinac_benchmarks)
deepcinac.cinac_benchmarks.do_traces_smoothing(traces)
deepcinac.cinac_benchmarks.benchmark_neuronal_activity_inferences(inferences_dir, results_path, colorfull_boxplots=True, white_background=False, color_cell_as_boxplot=False, with_legend=False, put_metric_as_y_axis_label=False, using_patch_for_legend=True, alpha_scatter=0.6, plot_proportion_frames_in_transients=False, with_cells=True, with_cell_number=True, predictions_stat_by_metrics=False, save_formats=['png', 'eps'])

Evaluate the performances of different inferences methods. Every directory in the main directory is considered a session with a ground truth and inferences to benchmark :param inferences_dir: directory containing other directories, one by session to benchmark :param results_path: :param colorfull_boxplots: if True boxplots are filled with color :param white_background: if True background is white, else it’s black :param with_cells: if True display scatter to represent metrics for a cell :param with_cell_number: Displau the cell number in the scatter if with_cells is True :param with_legend: if True, add legens to the F1 score plot

Returns:

deepcinac.cinac_benchmarks.extract_age(label)