OpenData.md 9.1 KB
====
SOFTWARE
NOTE: This is a list of Julia language packages that automate the loading process for specific datasets. To use the datasets you may use these packages or write your own Julia package or modify existing ones.
- CommonCrawl.jl :: Interface to common crawl dataset on Amazon S3.
- DataStreams.jl :: A Julia package to support streaming process of data .
- Faker.jl :: A package that generates fake data.
- FaceDatasets.jl :: A package for easy access to face-related datasets.
- PublicSuffix.jl :: Julia Interface for working with the Public Suffix List.
- RDatasets.jl :: Julia package for loading many of the data sets available in R.
- Socrata.jl :: An API wrapper for accessing the Socrata Open Data API and importing data into a DataFrame. Socrata is an open data platform used by many local and State governments as well as by the Federal Government in USA.
- UCIMLRepo.jl :: A small package to allow for easy access and download of datasets from UCI ML repository.
- WorldBankData.jl :: The World Bank provides free access to data about development at data.worldbank.org.
ACTUARIAL SCIENCE
Econometrics
- The FRED economic time series data.
ASTRONOMY
- NASA Open DataSets :: To solve looming challenges here on Earth using NASA data, tools, and resources.
AstroPhysics
- sndatasets :: Download and normalize published supernova photometric data.
BIOLOGY
- WP's list of biological databases.
Genome
- ChromosomeMappings :: This repository contains chromosome/contig name mappings between UCSC <-> Ensembl <-> Gencode for a variety of genomes.
- A database of OA/free RNA-seq and genomic data-sets used in research projects at JHU.
- Download Gene data (via ftp) which integrates information from a wide range of species. A record may include nomenclature, Reference Sequences (RefSeqs), maps, pathways, variations, phenotypes, and links to genome-, phenotype-, and locus-specific resources worldwide.
- Saccharomyces Genome Database
- Genome Project Database
- RefSeqGene defines genomic sequences to be used as reference standards for well-characterized genes and is part of the Locus Reference Genomic (LRG) Project.
- The 3000 Rice Genomes Project Data, GigaScience Database and Journal and blog article in BMC.
- NCBI's Sequence Read Archive (SRA)
- DataLad :: aims to provide access to scientific data available from various sources (e.g. lab or consortium web-sites such as Human connectome; data sharing portals such as OpenFMRI and CRCNS) through a single convenient interface and integrated with your software package managers (such as APT in Debian). Although initially targeting neuroimaging and neuroscience data in general, it will not be limited by the domain and a wide range of contributions are welcome. Get the source code on github.
Worms, Virus, Nematodes
- The central MANUELA database, a.k.a. _M_eiobenthic _A_nd _N_ematode biodiversity _U_nravelling _E_cological and _L_atitudinal _A_spects database is compiled by capturing the available data on meiobenthos on a broad European scale.
- Nematodes DB from the Blaxter Lab, based on analyses of ESTs or GSSs from "neglected taxa" using the PartiGene suite of programmes.
- Nematode Transcriptome Analyses.
- WormBase :: Species genomes with standardized sequence and annotations.
Genetics-Medicine
- NCBI Resources for Genetics and Medicine.
- HIV-1, Human Protein Interaction Database :: A database of known interactions of HIV-1 proteins with proteins from human hosts. It provides annotated bibliographies of published reports of protein interactions, with links to the corresponding PubMed records and sequence data.
Medical Imaging
- Computed Tomography Emphysema Database.
- Cornell's Public Medical Image Databases.
Molecular Biology
- SASBDB ::Small Angle Scattering Biological Data Bank.
Neuroscience
- Codeneuro-Datasets :: Shared data sets for collaborating, testing, and benchmarking.
- OpenfMRI.org :: A project dedicated to the free and open sharing of functional magnetic resonance imaging (fMRI) datasets, including raw data.
- Neuroscience Databases list.
- Neurovault :: A place where researchers can publicly store and share unthresholded statistical maps produced by MRI and PET studies.
Pharma
CHEMISTRY
- Chemical DB list on WP.
Crystallography
- Crystallographic databases list on WP.
- Protein Data Bank (PDB) on WP.
- Inorganic Crystal Structure Database
DATA
DATA-General
- awesome-public-datasets :: A collection of large-scale public datasets on the Internet.
- common-workflow-language :: Repository for CWL Specfications.
- datasets :: Original data or Aggregated / cleaned / restructured existing datasets. Released under Creative Commons Attribution-ShareAlike 4.0 International License.
- Freebase :: A community-curated database of well-known people, places, and things.
Wikidata :: A free linked database that acts as central storage for the structured data of its Wikimedia sister projects including Wikipedia, Wikivoyage, Wikisource, and others, that can be read and edited by both humans and machines.
- SPARQL endpoints to Wikidata with a general introduction to Wikidata and its data model.
World Bank Open Data :: Free and open access to data about development in countries around the globe.
DATA-Research
- Registry of Research Data Repositories :: provides researchers, funding organisations, libraries and publishers with over 1,000 listed research data repositories from all over the world making it the largest and most comprehensive online catalog of research data repositories on the web.
DATA-Scientific
- Scientific Databases list on WP.
Machine Learning
Math
- Juliaset.jl :: Generate Julia set images. This is created primarily as an example for JuliaBox hosted REST APIs.