Marine omics methods

These package currently supports FAIR-EASE pilot demos, but eventually it can serve for general purpose manipulation of metagenomic data, locally and in VREs.

The idea is to provide testable methods to allow as much flexibility and remixing of the functionalities provided.

EMO-BON

Specifically, we aim primarily to manipulate European Marine Omics Biodiversity Observation Network (EMO-BON) sampling data and metadata from ENA project PRJEB51688. The interactive dashboards and jupyter notebooks built on top of this repository can be found here.

The methods are mixture of statistical methods, plotting functionalities, metadata and data handling utilities and generators of holoviz panel widgets and panes. Experimental integration to Galaxy uses a wrapper around bioblend.

Other resources

FAIR-EASE jupyter-book on the marine omics pilot is also updated in parallel and with extra analysis resources

FAIR-EASE GitHub, website and linkedIn.

installation

Install from PyPI

Marine-omics-methods is available on PyPI and can be installed using pip.

# install manually UDAL data query layer, which is under development and not deployed to PyPI yet
pip install git+https://github.com/fair-ease/py-udal-mgo.git

pip install marine-omics

# or version for development
pip install marine-omics[all]

Install from source

Marine-omics-methods is available on GitHub and can be installed using pip.

pip install "marine-omics @ git+https://github.com/emo-bon/marine-omics-methods.git@main"

Or clone and pip install locally afterwards.

git clone https://github.com/emo-bon/marine-omics-methods.git
cd marine-omics-methods
pip install -e .

• API
  • Loading module
    • Parquet data
      • load_parquets()
      • load_parquets_udal()
    • Ro-crates
      • extract_all_datafiles()
      • extract_data_by_name()
      • get_rocrate_data()
      • get_rocrate_metadata_gh()
    • Utils
      • bytes_to_df()
  • Complex networks
    • build_interaction_graphs()
    • interaction_to_graph()
    • interaction_to_graph_with_pvals()
    • pairwise_jaccard_lower_triangle()
  • Constants
  • Diversity module
    • alpha_diversity_parametrized()
    • alpha_input()
    • beta_diversity_parametrized()
    • calculate_alpha_diversity()
    • calculate_shannon_index()
    • diversity_input()
    • find_taxa_in_table()
    • get_key_column()
    • run_permanova()
    • shannon_index()
    • update_subset_indicator()
    • update_taxa_count_indicator()
  • Galaxy integration
    • Gecco
      • Gecco.handle_create_history()
      • Gecco.handle_get_datasets()
      • Gecco.handle_get_histories()
      • Gecco.handle_login()
      • Gecco.handle_submit_gecco()
      • Gecco.handle_update_current_file_name()
      • Gecco.handle_update_current_history_id()
      • Gecco.handle_update_current_history_name()
      • Gecco.handle_upload_dataset()
    • RemGalaxy
      • RemGalaxy.clean_histories_for_display()
      • RemGalaxy.get_datasets()
      • RemGalaxy.get_datasets_by_key()
      • RemGalaxy.get_histories()
      • RemGalaxy.set_dataset()
      • RemGalaxy.set_galaxy_env()
      • RemGalaxy.set_history()
      • RemGalaxy.set_tool()
      • RemGalaxy.show_job_status()
      • RemGalaxy.upload_file()
  • Metadata module
    • clean_metadata()
    • enhance_metadata()
    • extract_season()
    • extract_season_single()
    • fill_na_for_object_columns()
    • filter_data()
    • filter_metadata_table()
    • filter_metadata_terms()
    • get_metadata()
    • get_metadata_udal()
    • merge_source_mat_id_to_data()
    • process_collection_date()
    • rename_metadata_terms_vre()
  • Panel dashboard module
    • close_server()
    • create_indicators_diversity()
    • create_indicators_landing_page()
    • diversity_select_widgets()
    • is_port_in_use()
    • serve_app()
    • tax_finder_selector()
  • Plotting module
    • Constants
    • alpha_plot()
    • av_alpha_plot()
    • beta_plot()
    • beta_plot_pc()
    • beta_plot_pc_granular()
    • change_legend_labels()
    • cut_xaxis_labels()
    • fold_legend_labels_from_series()
    • get_sankey()
    • hvplot_alpha_diversity()
    • hvplot_average_per_factor()
    • hvplot_bgcs_violin()
    • hvplot_heatmap()
    • hvplot_plot_pcoa_black()
    • mpl_alpha_diversity()
    • mpl_average_per_factor()
    • mpl_bgcs_violin()
    • mpl_plot_heatmap()
    • plot_domain_abundance()
    • plot_network()
    • plot_pcoa_black()
    • plot_tsne()
  • Statistical module
    • plot_association_histogram()
    • plot_fdr()
    • spearman_from_taxonomy()
  • Taxonomy module
    • aggregate_by_taxonomic_level()
    • clean_tax_row()
    • compute_bray_curtis()
    • fdr_pvals()
    • fill_taxonomy_placeholders()
    • logger
    • map_taxa_up()
    • normalize_abundance()
    • pivot_taxonomic_data()
    • prevalence_cutoff()
    • prevalence_cutoff_taxonomy()
    • rarefy_table()
    • remove_high_taxa()
    • separate_taxonomy()
    • separate_taxonomy_eukaryota()
    • split_metadata()
    • split_taxonomic_data()
    • split_taxonomic_data_pivoted()
    • split_taxonomy()
    • taxon_in_table()
  • Utilities of all sorts
    • check_index_names()
    • get_notebook_environment()
    • init_setup()
    • install_colab_packages()
    • is_ipython()
    • load_and_clean()
    • memory_load()
    • memory_usage()
    • reconfig_logger()
    • setup_ipython()
    • taxonomy_common_preprocess01()