List of datasets for machine-learning research
Part of a series on |
Machine learning and data mining |
---|
These datasets are used in machine learning (ML) research and have been cited in peer-reviewed academic journals. Datasets are an integral part of the field of machine learning. Major advances in this field can result from advances in learning algorithms (such as deep learning), computer hardware, and, less-intuitively, the availability of high-quality training datasets.[1] High-quality labeled training datasets for supervised and semi-supervised machine learning algorithms are usually difficult and expensive to produce because of the large amount of time needed to label the data. Although they do not need to be labeled, high-quality datasets for unsupervised learning can also be difficult and costly to produce.[2][3][4][5]
Many organizations including governments publish and share their datasets. The datasets are classified, based on the licenses, as Open data and Non-Open data.
The datasets from various governmental-bodies are presented in List of open government data sites. The datasets are ported on open data portals. They are made available for searching, depositing and accessing through interfaces like Open API. The datasets are made available as various sorted types and subtypes.
List of sorting used for datasets
[edit]Type | Subtypes |
---|---|
Specific category | Finance, Economics, Commerce, Societal, Health, Academy, Sports, Food, Agriculture, Travel, Geospatial, Political, Consumer, Transport, Logistics, Environmental, Real-Estate, Legal, Entertainment, Energy, Hospitality |
Scope | Supranational Union, National, Subnational, Municipality, Urban, Rural |
Language | Mandarin Chinese, Spanish, English, Arabic, Hindi, Bengali |
Type | Tabular, Graph, Text, Image, Sound, Video |
Usage | Training, validating, and testing |
File-Formats | CSV, JSON, XML, KML, GeoJSON, Shapefile, GML |
Licenses | Creative-Commons, GPL, Other Non-Open data licenses |
Last-Updated | Last-Hour, Last-Day, Last-Week, Last-Month, Last-Year |
File-Size | Minimum, Maximum, Range |
Status | Verified, In-Preparation, Deactivated(or Deprecated) |
Number of records | 100s, 1000s, 10000s, 100000s, Millions |
Number of variables | Less than 10, 10s, 100s, 1000s, 10000s |
Services | Individual, Aggregation |
The data portal is classified based on its type of license. The open source license based data portals are known as open data portals which are used by many government organizations and academic institutions.
List of open data portals
[edit]Portal-name | License | List of installations of the portal | Typical usages |
---|---|---|---|
Comprehensive Knowledge Archive Network (CKAN) | AGPL | https://ckan.github.io/ckan-instances/ https://github.com/sebneu/ckan_instances/blob/master/instances.csv | Data repository for government or non-profit organisations, Data Management Solution for Research Institutes |
DKAN | GPL | https://getdkan.org/community | Data repository for government or non-profit organisations, Data Management Solution for Research Institutes |
Dataverse | Apache | https://dataverse.org/installations | Data Management Solution for Research Institutes |
DSpace | BSD | https://registry.lyrasis.org/ | Data Management Solution for Research Institutes |
OpenML | BSD | https://www.openml.org/search?type=data&sort=runs&status=active | Data Management Solution to share datasets, algorithms, and experiments results through APIs. |
List of portals suitable for multiple types of applications
[edit]The data portal sometimes lists a wide variety of subtypes of datasets pertaining to many machine learning applications.
List of portals suitable for a specific subtype of applications
[edit]The data portals which are suitable for a specific subtype of machine learning application are listed in the subsequent sections.
Image data
[edit]Text data
[edit]These datasets consist primarily of text for tasks such as natural language processing, sentiment analysis, translation, and cluster analysis.
Reviews
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Amazon reviews | US product reviews from Amazon.com. | None. | 233.1 million | Text | Classification, sentiment analysis | 2015 (2018) | [6][7] | McAuley et al. |
OpinRank Review Dataset | Reviews of cars and hotels from Edmunds.com and TripAdvisor respectively. | None. | 42,230 / ~259,000 respectively | Text | Sentiment analysis, clustering | 2011 | [8][9] | K. Ganesan et al. |
MovieLens | 22,000,000 ratings and 580,000 tags applied to 33,000 movies by 240,000 users. | None. | ~ 22M | Text | Regression, clustering, classification | 2016 | [10] | GroupLens Research |
Yahoo! Music User Ratings of Musical Artists | Over 10M ratings of artists by Yahoo users. | None described. | ~ 10M | Text | Clustering, regression | 2004 | [11][12] | Yahoo! |
Car Evaluation Data Set | Car properties and their overall acceptability. | Six categorical features given. | 1728 | Text | Classification | 1997 | [13][14] | M. Bohanec |
YouTube Comedy Slam Preference Dataset | User vote data for pairs of videos shown on YouTube. Users voted on funnier videos. | Video metadata given. | 1,138,562 | Text | Classification | 2012 | [15][16] | |
Skytrax User Reviews Dataset | User reviews of airlines, airports, seats, and lounges from Skytrax. | Ratings are fine-grain and include many aspects of airport experience. | 41396 | Text | Classification, regression | 2015 | [17] | Q. Nguyen |
Teaching Assistant Evaluation Dataset | Teaching assistant reviews. | Features of each instance such as class, class size, and instructor are given. | 151 | Text | Classification | 1997 | [18][19] | W. Loh et al. |
Vietnamese Students’ Feedback Corpus (UIT-VSFC) | Students’ Feedback. | Comments | 16,000 | Text | Classification | 1997 | [20] | Nguyen et al. |
Vietnamese Social Media Emotion Corpus (UIT-VSMEC) | Users’ Facebook Comments. | Comments | 6,927 | Text | Classification | 1997 | [21] | Nguyen et al. |
Vietnamese Open-domain Complaint Detection dataset (ViOCD) | Customer product reviews | Comments | 5,485 | Text | Classification | 2021 | [22] | Nguyen et al. |
ViHOS: Hate Speech Spans Detection for Vietnamese | Social Media Texts | Comments | Containing 26k spans on 11k comments | Text | Span Detection | 2021 | [23] | Hoang et al. |
News articles
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
NYSK Dataset | English news articles about the case relating to allegations of sexual assault against the former IMF director Dominique Strauss-Kahn. | Filtered and presented in XML format. | 10,421 | XML, text | Sentiment analysis, topic extraction | 2013 | [24] | Dermouche, M. et al. |
The Reuters Corpus Volume 1 | Large corpus of Reuters news stories in English. | Fine-grain categorization and topic codes. | 810,000 | Text | Classification, clustering, summarization | 2002 | [25] | Reuters |
The Reuters Corpus Volume 2 | Large corpus of Reuters news stories in multiple languages. | Fine-grain categorization and topic codes. | 487,000 | Text | Classification, clustering, summarization | 2005 | [26] | Reuters |
Thomson Reuters Text Research Collection | Large corpus of news stories. | Details not described. | 1,800,370 | Text | Classification, clustering, summarization | 2009 | [27] | T. Rose et al. |
Saudi Newspapers Corpus | 31,030 Arabic newspaper articles. | Metadata extracted. | 31,030 | JSON | Summarization, clustering | 2015 | [28] | M. Alhagri |
RE3D (Relationship and Entity Extraction Evaluation Dataset) | Entity and Relation marked data from various news and government sources. Sponsored by Dstl | Filtered, categorisation using Baleen types | not known | JSON | Classification, Entity and Relation recognition | 2017 | [29] | Dstl |
Examiner Spam Clickbait Catalogue | Clickbait, spam, crowd-sourced headlines from 2010 to 2015 | Publish date and headlines | 3,089,781 | CSV | Clustering, Events, Sentiment | 2016 | [30] | R. Kulkarni |
ABC Australia News Corpus | Entire news corpus of ABC Australia from 2003 to 2019 | Publish date and headlines | 1,186,018 | CSV | Clustering, Events, Sentiment | 2020 | [31] | R. Kulkarni |
Worldwide News – Aggregate of 20K Feeds | One week snapshot of all online headlines in 20+ languages | Publish time, URL and headlines | 1,398,431 | CSV | Clustering, Events, Language Detection | 2018 | [32] | R. Kulkarni |
Reuters News Wire Headline | 11 Years of timestamped events published on the news-wire | Publish time, Headline Text | 16,121,310 | CSV | NLP, Computational Linguistics, Events | 2018 | [33] | R. Kulkarni |
The Irish Times Ireland News Corpus | 24 Years of Ireland News from 1996 to 2019 | Publish time, Headline Category and Text | 1,484,340 | CSV | NLP, Computational Linguistics, Events | 2020 | [34] | R. Kulkarni |
News Headlines Dataset for Sarcasm Detection | High quality dataset with Sarcastic and Non-sarcastic news headlines. | Clean, normalized text | 26,709 | JSON | NLP, Classification, Linguistics | 2018 | [35] | Rishabh Misra |
Messages
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Enron Email Dataset | Emails from employees at Enron organized into folders. | Attachments removed, invalid email addresses converted to [email protected] or [email protected]. | ~ 500,000 | Text | Network analysis, sentiment analysis | 2004 (2015) | [36][37] | Klimt, B. and Y. Yang |
Ling-Spam Dataset | Corpus containing both legitimate and spam emails. | Four version of the corpus involving whether or not a lemmatiser or stop-list was enabled. | 2,412 Ham 481 Spam | Text | Classification | 2000 | [38][39] | Androutsopoulos, J. et al. |
SMS Spam Collection Dataset | Collected SMS spam messages. | None. | 5,574 | Text | Classification | 2011 | [40][41] | T. Almeida et al. |
Twenty Newsgroups Dataset | Messages from 20 different newsgroups. | None. | 20,000 | Text | Natural language processing | 1999 | [42] | T. Mitchell et al. |
Spambase Dataset | Spam emails. | Many text features extracted. | 4,601 | Text | Spam detection, classification | 1999 | [43] | M. Hopkins et al. |
Twitter and tweets
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
MovieTweetings | Movie rating dataset based on public and well-structured tweets | ~710,000 | Text | Classification, regression | 2018 | [44] | S. Dooms | |
Twitter100k | Pairs of images and tweets | 100,000 | Text and Images | Cross-media retrieval | 2017 | [45][46] | Y. Hu, et al. | |
Sentiment140 | Tweet data from 2009 including original text, time stamp, user and sentiment. | Classified using distant supervision from presence of emoticon in tweet. | 1,578,627 | Tweets, comma, separated values | Sentiment analysis | 2009 | [47][48] | A. Go et al. |
ASU Twitter Dataset | Twitter network data, not actual tweets. Shows connections between a large number of users. | None. | 11,316,811 users, 85,331,846 connections | Text | Clustering, graph analysis | 2009 | [49][50] | R. Zafarani et al. |
SNAP Social Circles: Twitter Database | Large Twitter network data. | Node features, circles, and ego networks. | 1,768,149 | Text | Clustering, graph analysis | 2012 | [51][52] | J. McAuley et al. |
Twitter Dataset for Arabic Sentiment Analysis | Arabic tweets. | Samples hand-labeled as positive or negative. | 2000 | Text | Classification | 2014 | [53][54] | N. Abdulla |
Buzz in Social Media Dataset | Data from Twitter and Tom's Hardware. This dataset focuses on specific buzz topics being discussed on those sites. | Data is windowed so that the user can attempt to predict the events leading up to social media buzz. | 140,000 | Text | Regression, Classification | 2013 | [55][56] | F. Kawala et al. |
Paraphrase and Semantic Similarity in Twitter (PIT) | This dataset focuses on whether tweets have (almost) same meaning/information or not. Manually labeled. | tokenization, part-of-speech and named entity tagging | 18,762 | Text | Regression, Classification | 2015 | [57][58] | Xu et al. |
Geoparse Twitter benchmark dataset | This dataset contains tweets during different news events in different countries. Manually labeled location mentions. | location annotations added to JSON metadata | 6,386 | Tweets, JSON | Classification, Information Extraction | 2014 | [59][60] | S.E. Middleton et al. |
Sarcasm, Perceived and Intended, by Reactive Supervision (SPIRS) | Intended and perceived sarcastic tweets along with their context collected using reactive supervision; an equal number of negative (non-sarcastic) samples | 30,000 | Tweet IDs, CSV | Classification | 2020 | [61][62] | B. Shmueli et al. | |
Dutch Social media collection | This dataset contains COVID-19 tweets made by Dutch speakers or users from Netherlands. The data has been machine labeled | classified for sentiment, tweet text & user description translated to English. Industry mention are extracted | 271,342 | JSONL | Sentiment, multi-label classification, machine translation | 2020 | [63][64][65] | Aaaksh Gupta, CoronaWhy |
ReactionGIF dataset | A dataset of 30K tweets and their GIF reactions | Classified for sentiment, reaction, and emotion | 30,000 | Tweet IDs, JSONL | Classified for sentiment, reaction, and emotion | 2021 | [66][67] | B. Shmueli et al. |
Dialogues
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
NPS Chat Corpus | Posts from age-specific online chat rooms. | Hand privacy masked, tagged for part of speech and dialogue-act. | ~ 500,000 | XML | NLP, programming, linguistics | 2007 | [68] | Forsyth, E., Lin, J., & Martell, C. |
Twitter Triple Corpus | A-B-A triples extracted from Twitter. | 4,232 | Text | NLP | 2016 | [69] | Sordini, A. et al. | |
UseNet Corpus | UseNet forum postings. | Anonymized e-mails and URLs. Omitted documents with lengths <500 words or >500,000 words, or that were <90% English. | 7 billion | Text | 2011 | [70] | Shaoul, C., & Westbury C. | |
NUS SMS Corpus | SMS messages collected between two users, with timing analysis. | ~ 10,000 | XML | NLP | 2011 | [71] | KAN, M | |
Reddit All Comments Corpus | All Reddit comments (as of 2015). | ~ 1.7 billion | JSON | NLP, research | 2015 | [72] | Stuck_In_the_Matrix | |
Ubuntu Dialogue Corpus | Dialogues extracted from Ubuntu chat stream on IRC. | 930 thousand dialogues, 7.1 million utterances | CSV | Dialogue Systems Research | 2015 | [73] | Lowe, R. et al. | |
Dialog State Tracking Challenge | The Dialog State Tracking Challenges 2 & 3 (DSTC2&3) were research challenge focused on improving the state of the art in tracking the state of spoken dialog systems. | Transcription of spoken dialogs with labelling | DSTC2 contains ~3.2k calls – DSTC3 contains ~2.3k calls | Json | Dialogue state tracking | 2014 | [74] | Henderson, Matthew and Thomson, Blaise and Williams, Jason D |
Legal
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
FreeLaw | Filtered data from Court Listener, part of the FreeLaw project. | Cleaned and normalized text | 4,940,710 | Json | NLP, linguistics | 2020 | [75] | T. Hoppe |
Pile of Law | Corpus of legal and administrative data | Cleaned, normalized, and privatized | ~50,000,000 | Json | NLP, linguistics, sentiment | 2022 | [76][77] | L. Zheng; N. Guha; B. Anderson; P. Henderson; D. Ho |
Caselaw Access Project | All official, book-published state and federal United States case law — every volume or case designated as an official report of decisions by a court within the United States. | Cleaned and normalized text | ~10,000 | Json | NLP, linguistics | 2022 | [78] | A. Aizman; S. Chapman; J. Cushman; K. Dulin; H. Eidolon; et al. |
Other text
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Web of Science Dataset | Hierarchical Datasets for Text Classification | None. | 46,985 | Text | Classification, Categorization | 2017 | [79][80] | K. Kowsari et al. |
Legal Case Reports | Federal Court of Australia cases from 2006 to 2009. | None. | 4,000 | Text | Summarization, citation analysis | 2012 | [81][82] | F. Galgani et al. |
Blogger Authorship Corpus | Blog entries of 19,320 people from blogger.com. | Blogger self-provided gender, age, industry, and astrological sign. | 681,288 | Text | Sentiment analysis, summarization, classification | 2006 | [83][84] | J. Schler et al. |
Social Structure of Facebook Networks | Large dataset of the social structure of Facebook. | None. | 100 colleges covered | Text | Network analysis, clustering | 2012 | [85][86] | A. Traud et al. |
Dataset for the Machine Comprehension of Text | Stories and associated questions for testing comprehension of text. | None. | 660 | Text | Natural language processing, machine comprehension | 2013 | [87][88] | M. Richardson et al. |
The Penn Treebank Project | Naturally occurring text annotated for linguistic structure. | Text is parsed into semantic trees. | ~ 1M words | Text | Natural language processing, summarization | 1995 | [89][90] | M. Marcus et al. |
DEXTER Dataset | Task given is to determine, from features given, which articles are about corporate acquisitions. | Features extracted include word stems. Distractor features included. | 2600 | Text | Classification | 2008 | [91] | Reuters |
Google Books N-grams | N-grams from a very large corpus of books | None. | 2.2 TB of text | Text | Classification, clustering, regression | 2011 | [92][93] | |
Personae Corpus | Collected for experiments in Authorship Attribution and Personality Prediction. Consists of 145 Dutch-language essays. | In addition to normal texts, syntactically annotated texts are given. | 145 | Text | Classification, regression | 2008 | [94][95] | K. Luyckx et al. |
PushShift | Archives of social media websites, including Reddit, Twitter, and Hackernews. | Text extracted and normalized from WARCs | ~100,000,000 posts | Json | NLP, sentiment, linguistics | 2022 | [96][97] | J. Baumgartner |
SEC Filings | EDGAR | Company Filings | Text extracted. | csv | NLP | ||||
CNAE-9 Dataset | Categorization task for free text descriptions of Brazilian companies. | Word frequency has been extracted. | 1080 | Text | Classification | 2012 | [98][99] | P. Ciarelli et al. |
Sentiment Labeled Sentences Dataset | 3000 sentiment labeled sentences. | Sentiment of each sentence has been hand labeled as positive or negative. | 3000 | Text | Classification, sentiment analysis | 2015 | [100][101] | D. Kotzias |
BlogFeedback Dataset | Dataset to predict the number of comments a post will receive based on features of that post. | Many features of each post extracted. | 60,021 | Text | Regression | 2014 | [102][103] | K. Buza |
PubMed Central | PubMed® comprises more than 35 million citations for biomedical literature from MEDLINE, life science journals, and online books. | None | 35 Million | Text | NLP | |||
USPTO | The United States Patent and Trademark Office | Text | NLP | |||||
PhilPapers | Open access collection of philosophy publications | Text | NLP | |||||
Book Corpus | A popular large-scale text corpus. | None | Text | NLP | 2015 | [104] | Zhu, Yukun, et al. | |
Stanford Natural Language Inference (SNLI) Corpus | Image captions matched with newly constructed sentences to form entailment, contradiction, or neutral pairs. | Entailment class labels, syntactic parsing by the Stanford PCFG parser | 570,000 | Text | Natural language inference/recognizing textual entailment | 2015 | [105] | S. Bowman et al. |
DSL Corpus Collection (DSLCC) | A multilingual collection of short excerpts of journalistic texts in similar languages and dialects. | None | 294,000 phrases | Text | Discriminating between similar languages | 2017 | [106] | Tan, Liling et al. |
Urban Dictionary Dataset | Corpus of words, votes and definitions | User names anonymised | 2,580,925 | CSV | NLP, Machine comprehension | 2016 May | [107] | Anonymous |
T-REx | Wikipedia abstracts aligned with Wikidata entities | Alignment of Wikidata triples with Wikipedia abstracts | 11M aligned triples | JSON and NIF [4] | NLP, Relation Extraction | 2018 | [108] | H. Elsahar et al. |
General Language Understanding Evaluation (GLUE) | Benchmark of nine tasks | Various | ~1M sentences and sentence pairs | NLU | 2018 | [109][110][111] | Wang et al. | |
Contract Understanding Atticus Dataset (CUAD) (formerly known as Atticus Open Contract Dataset (AOK)) | Dataset of legal contracts with rich expert annotations | ~13,000 labels | CSV and PDF | Natural language processing, QnA | 2021 | The Atticus Project | ||
Vietnamese Image Captioning Dataset (UIT-ViIC) | Vietnamese Image Captioning Dataset | 19,250 captions for 3,850 images | CSV and PDF | Natural language processing, Computer vision | 2020 | [112] | Lam et al. | |
Vietnamese Names annotated with Genders (UIT-ViNames) | Vietnamese Names annotated with Genders | 26,850 Vietnamese full names annotated with genders | CSV | Natural language processing | 2020 | [113] | To et al. | |
Vietnamese Constructive and Toxic Speech Detection Dataset (UIT-ViCTSD) | Vietnamese Constructive and Toxic Speech Detection Dataset | 10,000 Vietnamese users' comments on online newspapers on 10 domains | CSV | Natural Language Processing | 2021 | [114] | Nguyen et al. | |
PG-19 | A set of books extracted from the Project Gutenberg books library | Text | Natural Language Processing | 2019 | Jack W et al. | |||
Deepmind Mathematics | Mathematical question and answer pairs. | Text | Natural Language Processing | 2018 | [115] | D Saxton et al. | ||
Anna's Archive | A comprehensive archive of published books and papers | None | 100,356,641 | Text, epub, PDF | Natural Language Processing | 2024 |
Sound data
[edit]These datasets consist of sounds and sound features used for tasks such as speech recognition and speech synthesis.
Speech
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Zero Resource Speech Challenge 2015 | Spontaneous speech (English), Read speech (Xitsonga). | None, raw WAV files. | English: 5h, 12 speakers; Xitsonga: 2h30, 24 speakers | WAV (audio only) | Unsupervised discovery of speech features/subword units/word units | 2015 | [116][117] | Versteegh et al. |
Parkinson Speech Dataset | Multiple recordings of people with and without Parkinson's Disease. | Voice features extracted, disease scored by physician using unified Parkinson's disease rating scale. | 1,040 | Text | Classification, regression | 2013 | [118][119] | B. E. Sakar et al. |
Spoken Arabic Digits | Spoken Arabic digits from 44 male and 44 female. | Time-series of mel-frequency cepstrum coefficients. | 8,800 | Text | Classification | 2010 | [120][121] | M. Bedda et al. |
ISOLET Dataset | Spoken letter names. | Features extracted from sounds. | 7797 | Text | Classification | 1994 | [122][123] | R. Cole et al. |
Japanese Vowels Dataset | Nine male speakers uttered two Japanese vowels successively. | Applied 12-degree linear prediction analysis to it to obtain a discrete-time series with 12 cepstrum coefficients. | 640 | Text | Classification | 1999 | [124][125] | M. Kudo et al. |
Parkinson's Telemonitoring Dataset | Multiple recordings of people with and without Parkinson's Disease. | Sound features extracted. | 5875 | Text | Classification | 2009 | [126][127] | A. Tsanas et al. |
TIMIT | Recordings of 630 speakers of eight major dialects of American English, each reading ten phonetically rich sentences. | Speech is lexically and phonemically transcribed. | 6300 | Text | Speech recognition, classification. | 1986 | [128][129] | J. Garofolo et al. |
Arabic Speech Corpus | A single-speaker, Modern Standard Arabic (MSA) speech corpus with phonetic and orthographic transcripts aligned to phoneme level. | Speech is orthographically and phonetically transcribed with stress marks. | ~1900 | Text, WAV | Speech Synthesis, Speech Recognition, Corpus Alignment, Speech Therapy, Education. | 2016 | [130] | N. Halabi |
Common Voice | A public domain database of crowdsourced data across a wide range of dialects. | Validation by other users . | English: 1,118 hours | MP3 with corresponding text files | Speech recognition | 2017 June (2019 December) | [131] | Mozilla |
LJSpeech | A single-speaker corpus of English public-domain audiobook recordings, split into short clips at punctuation marks. | Quality check, normalized transcription alongside the original. | 13,100 | CSV, WAV | Speech synthesis | 2017 | [132] | Keith Ito, Linda Johnson |
Arabic Speech Commands Dataset | Collected from 30 contributors and grouped into 40 keywords. | Raw WAV files | 12,000 | WAV, CSV | Speech recognition, keyword spotting | 2021 | [133] | Abdulkader Ghandoura |
Music
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Geographic Origin of Music Data Set | Audio features of music samples from different locations. | Audio features extracted using MARSYAS software. | 1,059 | Text | Geographic classification, clustering | 2014 | [134][135] | F. Zhou et al. |
Million Song Dataset | Audio features from one million different songs. | Audio features extracted. | 1M | Text | Classification, clustering | 2011 | [136][137] | T. Bertin-Mahieux et al. |
MUSDB18 | Multi-track popular music recordings | Raw audio | 150 | MP4, WAV | Source Separation | 2017 | [138] | Z. Rafii et al. |
Free Music Archive | Audio under Creative Commons from 100k songs (343 days, 1TiB) with a hierarchy of 161 genres, metadata, user data, free-form text. | Raw audio and audio features. | 106,574 | Text, MP3 | Classification, recommendation | 2017 | [139] | M. Defferrard et al. |
Bach Choral Harmony Dataset | Bach chorale chords. | Audio features extracted. | 5665 | Text | Classification | 2014 | [140][141] | D. Radicioni et al. |
Other sounds
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
UrbanSound | Labeled sound recordings of sounds like air conditioners, car horns and children playing. | Sorted into folders by class of events as well as metadata in a JSON file and annotations in a CSV file. | 1,059 | Sound (WAV) | Classification | 2014 | [142][143] | J. Salamon et al. |
AudioSet | 10-second sound snippets from YouTube videos, and an ontology of over 500 labels. | 128-d PCA'd VGG-ish features every 1 second. | 2,084,320 | Text (CSV) and TensorFlow Record files | Classification | 2017 | [144] | J. Gemmeke et al., Google |
Bird Audio Detection challenge | Audio from environmental monitoring stations, plus crowdsourced recordings | 17,000+ | Classification | 2016 (2018) | [145][146] | Queen Mary University and IEEE Signal Processing Society | ||
WSJ0 Hipster Ambient Mixtures | Audio from WSJ0 mixed with noise recorded in the San Francisco Bay Area | Noise clips matched to WSJ0 clips | 28,000 | Sound (WAV) | Audio source separation | 2019 | [147] | Wichern, G., et al., Whisper and MERL |
Clotho | 4,981 audio samples of 15 to 30 seconds long, each audio sample having five different captions of eight to 20 words long. | 24,905 | Sound (WAV) and text (CSV) | Automated audio captioning | 2020 | [148][149] | K. Drossos, S. Lipping, and T. Virtanen |
Signal data
[edit]Datasets containing electric signal information requiring some sort of signal processing for further analysis.
Electrical
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Witty Worm Dataset | Dataset detailing the spread of the Witty worm and the infected computers. | Split into a publicly available set and a restricted set containing more sensitive information like IP and UDP headers. | 55,909 IP addresses | Text | Classification | 2004 | [150][151] | Center for Applied Internet Data Analysis |
Cuff-Less Blood Pressure Estimation Dataset | Cleaned vital signals from human patients which can be used to estimate blood pressure. | 125 Hz vital signs have been cleaned. | 12,000 | Text | Classification, regression | 2015 | [152][153] | M. Kachuee et al. |
Gas Sensor Array Drift Dataset | Measurements from 16 chemical sensors utilized in simulations for drift compensation. | Extensive number of features given. | 13,910 | Text | Classification | 2012 | [154][155] | A. Vergara |
Servo Dataset | Data covering the nonlinear relationships observed in a servo-amplifier circuit. | Levels of various components as a function of other components are given. | 167 | Text | Regression | 1993 | [156][157] | K. Ullrich |
UJIIndoorLoc-Mag Dataset | Indoor localization database to test indoor positioning systems. Data is magnetic field based. | Train and test splits given. | 40,000 | Text | Classification, regression, clustering | 2015 | [158][159] | D. Rambla et al. |
Sensorless Drive Diagnosis Dataset | Electrical signals from motors with defective components. | Statistical features extracted. | 58,508 | Text | Classification | 2015 | [160][161] | M. Bator |
Motion-tracking
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Wearable Computing: Classification of Body Postures and Movements (PUC-Rio) | People performing five standard actions while wearing motion trackers. | None. | 165,632 | Text | Classification | 2013 | [162][163] | Pontifical Catholic University of Rio de Janeiro |
Gesture Phase Segmentation Dataset | Features extracted from video of people doing various gestures. | Features extracted aim at studying gesture phase segmentation. | 9900 | Text | Classification, clustering | 2014 | [164][165] | R. Madeo et a |
Vicon Physical Action Data Set Dataset | 10 normal and 10 aggressive physical actions that measure the human activity tracked by a 3D tracker. | Many parameters recorded by 3D tracker. | 3000 | Text | Classification | 2011 | [166][167] | T. Theodoridis |
Daily and Sports Activities Dataset | Motor sensor data for 19 daily and sports activities. | Many sensors given, no preprocessing done on signals. | 9120 | Text | Classification | 2013 | [168][169] | B. Barshan et al. |
Human Activity Recognition Using Smartphones Dataset | Gyroscope and accelerometer data from people wearing smartphones and performing normal actions. | Actions performed are labeled, all signals preprocessed for noise. | 10,299 | Text | Classification | 2012 | [170][171] | J. Reyes-Ortiz et al. |
Australian Sign Language Signs | Australian sign language signs captured by motion-tracking gloves. | None. | 2565 | Text | Classification | 2002 | [172][173] | M. Kadous |
Weight Lifting Exercises monitored with Inertial Measurement Units | Five variations of the biceps curl exercise monitored with IMUs. | Some statistics calculated from raw data. | 39,242 | Text | Classification | 2013 | [174][175] | W. Ugulino et al. |
sEMG for Basic Hand movements Dataset | Two databases of surface electromyographic signals of 6 hand movements. | None. | 3000 | Text | Classification | 2014 | [176][177] | C. Sapsanis et al. |
REALDISP Activity Recognition Dataset | Evaluate techniques dealing with the effects of sensor displacement in wearable activity recognition. | None. | 1419 | Text | Classification | 2014 | [177][178] | O. Banos et al. |
Heterogeneity Activity Recognition Dataset | Data from multiple different smart devices for humans performing various activities. | None. | 43,930,257 | Text | Classification, clustering | 2015 | [179][180] | A. Stisen et al. |
Indoor User Movement Prediction from RSS Data | Temporal wireless network data that can be used to track the movement of people in an office. | None. | 13,197 | Text | Classification | 2016 | [181][182] | D. Bacciu |
PAMAP2 Physical Activity Monitoring Dataset | 18 different types of physical activities performed by 9 subjects wearing 3 IMUs. | None. | 3,850,505 | Text | Classification | 2012 | [183] | A. Reiss |
OPPORTUNITY Activity Recognition Dataset | Human Activity Recognition from wearable, object, and ambient sensors is a dataset devised to benchmark human activity recognition algorithms. | None. | 2551 | Text | Classification | 2012 | [184][185] | D. Roggen et al. |
Real World Activity Recognition Dataset | Human Activity Recognition from wearable devices. Distinguishes between seven on-body device positions and comprises six different kinds of sensors. | None. | 3,150,000 (per sensor) | Text | Classification | 2016 | [186] | T. Sztyler et al. |
Toronto Rehab Stroke Pose Dataset | 3D human pose estimates (Kinect) of stroke patients and healthy participants performing a set of tasks using a stroke rehabilitation robot. | None. | 10 healthy person and 9 stroke survivors (3500–6000 frames per person) | CSV | Classification | 2017 | [187][188][189] | E. Dolatabadi et al. |
Corpus of Social Touch (CoST) | 7805 gesture captures of 14 different social touch gestures performed by 31 subjects. The gestures were performed in three variations: gentle, normal and rough, on a pressure sensor grid wrapped around a mannequin arm. | Touch gestures performed are segmented and labeled. | 7805 gesture captures | CSV | Classification | 2016 | [190][191] | M. Jung et al. |
Other signals
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Wine Dataset | Chemical analysis of wines grown in the same region in Italy but derived from three different cultivars. | 13 properties of each wine are given | 178 | Text | Classification, regression | 1991 | [192][193] | M. Forina et al. |
Combined Cycle Power Plant Data Set | Data from various sensors within a power plant running for 6 years. | None | 9568 | Text | Regression | 2014 | [194][195] | P. Tufekci et al. |
Physical data
[edit]Datasets from physical systems.
High-energy physics
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
HIGGS Dataset | Monte Carlo simulations of particle accelerator collisions. | 28 features of each collision are given. | 11M | Text | Classification | 2014 | [196][197][198] | D. Whiteson |
HEPMASS Dataset | Monte Carlo simulations of particle accelerator collisions. Goal is to separate the signal from noise. | 28 features of each collision are given. | 10,500,000 | Text | Classification | 2016 | [197][198][199] | D. Whiteson |
Systems
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Yacht Hydrodynamics Dataset | Yacht performance based on dimensions. | Six features are given for each yacht. | 308 | Text | Regression | 2013 | [200][201] | R. Lopez |
Robot Execution Failures Dataset | 5 data sets that center around robotic failure to execute common tasks. | Integer valued features such as torque and other sensor measurements. | 463 | Text | Classification | 1999 | [202] | L. Seabra et al. |
Pittsburgh Bridges Dataset | Design description is given in terms of several properties of various bridges. | Various bridge features are given. | 108 | Text | Classification | 1990 | [203][204] | Y. Reich et al. |
Automobile Dataset | Data about automobiles, their insurance risk, and their normalized losses. | Car features extracted. | 205 | Text | Regression | 1987 | [205][206] | J. Schimmer et al. |
Auto MPG Dataset | MPG data for cars. | Eight features of each car given. | 398 | Text | Regression | 1993 | [207] | Carnegie Mellon University |
Energy Efficiency Dataset | Heating and cooling requirements given as a function of building parameters. | Building parameters given. | 768 | Text | Classification, regression | 2012 | [208][209] | A. Xifara et al. |
Airfoil Self-Noise Dataset | A series of aerodynamic and acoustic tests of two and three-dimensional airfoil blade sections. | Data about frequency, angle of attack, etc., are given. | 1503 | Text | Regression | 2014 | [210] | R. Lopez |
Challenger USA Space Shuttle O-Ring Dataset | Attempt to predict O-ring problems given past Challenger data. | Several features of each flight, such as launch temperature, are given. | 23 | Text | Regression | 1993 | [211][212] | D. Draper et al. |
Statlog (Shuttle) Dataset | NASA space shuttle datasets. | Nine features given. | 58,000 | Text | Classification | 2002 | [213] | NASA |
Astronomy
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Volcanoes on Venus – JARtool experiment Dataset | Venus images returned by the Magellan spacecraft. | Images are labeled by humans. | not given | Images | Classification | 1991 | [214][215] | M. Burl |
MAGIC Gamma Telescope Dataset | Monte Carlo generated high-energy gamma particle events. | Numerous features extracted from the simulations. | 19,020 | Text | Classification | 2007 | [215][216] | R. Bock |
Solar Flare Dataset | Measurements of the number of certain types of solar flare events occurring in a 24-hour period. | Many solar flare-specific features are given. | 1389 | Text | Regression, classification | 1989 | [217] | G. Bradshaw |
CAMELS Multifield Dataset | 2D maps and 3D grids from thousands of N-body and state-of-the-art hydrodynamic simulations spanning a broad range in the value of the cosmological and astrophysical parameters | Each map and grid has 6 cosmological and astrophysical parameters associated to it | 405,000 2D maps and 405,000 3D grids | 2D maps and 3D grids | Regression | 2021 | [218] | Francisco Villaescusa-Navarro et al. |
Earth science
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Volcanoes of the World | Volcanic eruption data for all known volcanic events on earth. | Details such as region, subregion, tectonic setting, dominant rock type are given. | 1535 | Text | Regression, classification | 2013 | [219] | E. Venzke et al. |
Seismic-bumps Dataset | Seismic activities from a coal mine. | Seismic activity was classified as hazardous or not. | 2584 | Text | Classification | 2013 | [220][221] | M. Sikora et al. |
CAMELS-US | Catchment hydrology dataset with hydrometeorological timeseries and various attributes | see Reference | 671 | CSV, Text, Shapefile | Regression | 2017 | [222][223] | N. Addor et al. / A. Newman et al. |
CAMELS-Chile | Catchment hydrology dataset with hydrometeorological timeseries and various attributes | see Reference | 516 | CSV, Text, Shapefile | Regression | 2018 | [224] | C. Alvarez-Garreton et al. |
CAMELS-Brazil | Catchment hydrology dataset with hydrometeorological timeseries and various attributes | see Reference | 897 | CSV, Text, Shapefile | Regression | 2020 | [225] | V. Chagas et al. |
CAMELS-GB | Catchment hydrology dataset with hydrometeorological timeseries and various attributes | see Reference | 671 | CSV, Text, Shapefile | Regression | 2020 | [226] | G. Coxon et al. |
CAMELS-Australia | Catchment hydrology dataset with hydrometeorological timeseries and various attributes | see Reference | 222 | CSV, Text, Shapefile | Regression | 2021 | [227] | K. Fowler et al. |
LamaH-CE | Catchment hydrology dataset with hydrometeorological timeseries and various attributes | see Reference | 859 | CSV, Text, Shapefile | Regression | 2021 | [228] | C. Klingler et al. |
Other physical
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Concrete Compressive Strength Dataset | Dataset of concrete properties and compressive strength. | Nine features are given for each sample. | 1030 | Text | Regression | 2007 | [229][230] | I. Yeh |
Concrete Slump Test Dataset | Concrete slump flow given in terms of properties. | Features of concrete given such as fly ash, water, etc. | 103 | Text | Regression | 2009 | [231][232] | I. Yeh |
Musk Dataset | Predict if a molecule, given the features, will be a musk or a non-musk. | 168 features given for each molecule. | 6598 | Text | Classification | 1994 | [233] | Arris Pharmaceutical Corp. |
Steel Plates Faults Dataset | Steel plates of 7 different types. | 27 features given for each sample. | 1941 | Text | Classification | 2010 | [234] | Semeion Research Center |
Noble Metal Monometallic Nanoparticles Datasets | Processing and structural features of monometallic nanoparticles, labels being formation energy. | 85-182 features given for each sample. | 425 to 4000 | CSV | Regression | 2017 to 2023 | [235][236][237][238][239][240] | A. Barnard and G. Opletal |
Noble Metal Bimetallic Nanoparticles Datasets | Processing and structural features of bimetallic nanoparticles, labels being formation energy. | 922 features given for each sample. | 138147 to 162770 | CSV | Regression | 2023 | [241][242][243][244][245][246][247][248][249][250][251][252] | J. Ting et al. |
AuPdPt Trimetallic Nanoparticles Dataset | Processing and structural features of AuPdPt nanoparticles, labels being formation energy. | 1958 features given for each sample. | 48136 | CSV | Regression | 2023 | [253] | K. Lu et al. |
Biological data
[edit]Datasets from biological systems.
Human
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Age Dataset | A structured general-purpose dataset on life, work, and death of 1.22 million distinguished people. Public domain. | A five-step method to infer birth and death years, gender, and occupation from community-submitted data to all language versions of the Wikipedia project. | 1,223,009 | Text | Regression, Classification | 2022 | Paper[254] Dataset[255] | Amoradnejad et al. |
Synthetic Fundus Dataset[256] | Photorealistic retinal images and vessel segmentations. Public domain. | 2500 images with 1500*1152 pixels useful for segmentation and classification of veins and arteries on a single background. | 2500 | Images | Classification, Segmentation | 2020 | [257] | C. Valenti et al. |
EEG Database | Study to examine EEG correlates of genetic predisposition to alcoholism. | Measurements from 64 electrodes placed on the scalp sampled at 256 Hz (3.9 ms epoch) for 1 second. | 122 | Text | Classification | 1999 | [258] | H. Begleiter |
P300 Interface Dataset | Data from nine subjects collected using P300-based brain-computer interface for disabled subjects. | Split into four sessions for each subject. MATLAB code given. | 1,224 | Text | Classification | 2008 | [259][260] | U. Hoffman et al. |
Heart Disease Data Set | Attributed of patients with and without heart disease. | 75 attributes given for each patient with some missing values. | 303 | Text | Classification | 1988 | [261][262] | A. Janosi et al. |
Breast Cancer Wisconsin (Diagnostic) Dataset | Dataset of features of breast masses. Diagnoses by physician is given. | 10 features for each sample are given. | 569 | Text | Classification | 1995 | [263][264] | W. Wolberg et al. |
National Survey on Drug Use and Health | Large scale survey on health and drug use in the United States. | None. | 55,268 | Text | Classification, regression | 2012 | [265] | United States Department of Health and Human Services |
Lung Cancer Dataset | Lung cancer dataset without attribute definitions | 56 features are given for each case | 32 | Text | Classification | 1992 | [266][267] | Z. Hong et al. |
Arrhythmia Dataset | Data for a group of patients, of which some have cardiac arrhythmia. | 276 features for each instance. | 452 | Text | Classification | 1998 | [268][269] | H. Altay et al. |
Diabetes 130-US hospitals for years 1999–2008 Dataset | 9 years of readmission data across 130 US hospitals for patients with diabetes. | Many features of each readmission are given. | 100,000 | Text | Classification, clustering | 2014 | [270][271] | J. Clore et al. |
Diabetic Retinopathy Debrecen Dataset | Features extracted from images of eyes with and without diabetic retinopathy. | Features extracted and conditions diagnosed. | 1151 | Text | Classification | 2014 | [272][273] | B. Antal et al. |
Diabetic Retinopathy Messidor Dataset | Methods to evaluate segmentation and indexing techniques in the field of retinal ophthalmology (MESSIDOR) | Features retinopathy grade and risk of macular edema | 1200 | Images, Text | Classification, Segmentation | 2008 | [274][275] | Messidor Project |
Liver Disorders Dataset | Data for people with liver disorders. | Seven biological features given for each patient. | 345 | Text | Classification | 1990 | [276][277] | Bupa Medical Research Ltd. |
Thyroid Disease Dataset | 10 databases of thyroid disease patient data. | None. | 7200 | Text | Classification | 1987 | [278][279] | R. Quinlan |
Mesothelioma Dataset | Mesothelioma patient data. | Large number of features, including asbestos exposure, are given. | 324 | Text | Classification | 2016 | [280][281] | A. Tanrikulu et al. |
Parkinson's Vision-Based Pose Estimation Dataset | 2D human pose estimates of Parkinson's patients performing a variety of tasks. | Camera shake has been removed from trajectories. | 134 | Text | Classification, regression | 2017 | [282][283][284] | M. Li et al. |
KEGG Metabolic Reaction Network (Undirected) Dataset | Network of metabolic pathways. A reaction network and a relation network are given. | Detailed features for each network node and pathway are given. | 65,554 | Text | Classification, clustering, regression | 2011 | [285] | M. Naeem et al. |
Modified Human Sperm Morphology Analysis Dataset (MHSMA) | Human sperm images from 235 patients with male factor infertility, labeled for normal or abnormal sperm acrosome, head, vacuole, and tail. | Cropped around single sperm head. Magnification normalized. Training, validation, and test set splits created. | 1,540 | .npy files | Classification | 2019 | [286][287] | S. Javadi and S.A. Mirroshandel |
Animal
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Abalone Dataset | Physical measurements of Abalone. Weather patterns and location are also given. | None. | 4177 | Text | Regression | 1995 | [288] | Marine Research Laboratories – Taroona |
Zoo Dataset | Artificial dataset covering 7 classes of animals. | Animals are classed into 7 categories and features are given for each. | 101 | Text | Classification | 1990 | [289] | R. Forsyth |
Demospongiae Dataset | Data about marine sponges. | 503 sponges in the Demosponge class are described by various features. | 503 | Text | Classification | 2010 | [290] | E. Armengol et al. |
Farm animals data | PLF data inventory (cows, pigs; location, acceleration, etc.). | Labeled datasets. | List is constantly updated | Text | Classification | 2020 | [291] | V. Bloch |
Splice-junction Gene Sequences Dataset | Primate splice-junction gene sequences (DNA) with associated imperfect domain theory. | None. | 3190 | Text | Classification | 1992 | [267] | G. Towell et al. |
Mice Protein Expression Dataset | Expression levels of 77 proteins measured in the cerebral cortex of mice. | None. | 1080 | Text | Classification, Clustering | 2015 | [292][293] | C. Higuera et al. |
Fungi
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
UCI Mushroom Dataset | Mushroom attributes and classification. | Many properties of each mushroom are given. | 8124 | Text | Classification | 1987 | [294] | J. Schlimmer |
Secondary Mushroom Dataset | Mushroom attributes and classification | Simulated data from larger and more realistic primary mushroom entries. Fully reproducible. | 61069 | Text | Classification | 2020 | [295][296] | D. Wagner et al. |
Plant
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Forest Fires Dataset | Forest fires and their properties. | 13 features of each fire are extracted. | 517 | Text | Regression | 2008 | [297][298] | P. Cortez et al. |
Iris Dataset | Three types of iris plants are described by 4 different attributes. | None. | 150 | Text | Classification | 1936 | [299][300] | R. Fisher |
Plant Species Leaves Dataset | Sixteen samples of leaf each of one-hundred plant species. | Shape descriptor, fine-scale margin, and texture histograms are given. | 1600 | Text | Classification | 2012 | [301][302] | J. Cope et al. |
Soybean Dataset | Database of diseased soybean plants. | 35 features for each plant are given. Plants are classified into 19 categories. | 307 | Text | Classification | 1988 | [303] | R. Michalski et al. |
Seeds Dataset | Measurements of geometrical properties of kernels belonging to three different varieties of wheat. | None. | 210 | Text | Classification, clustering | 2012 | [304][305] | Charytanowicz et al. |
Covertype Dataset | Data for predicting forest cover type strictly from cartographic variables. | Many geographical features given. | 581,012 | Text | Classification | 1998 | [306][307] | J. Blackard et al. |
Abscisic Acid Signaling Network Dataset | Data for a plant signaling network. Goal is to determine set of rules that governs the network. | None. | 300 | Text | Causal-discovery | 2008 | [308] | J. Jenkens et al. |
Folio Dataset | 20 photos of leaves for each of 32 species. | None. | 637 | Images, text | Classification, clustering | 2015 | [309][310] | T. Munisami et al. |
Oxford Flower Dataset | 17 category dataset of flowers. | Train/test splits, labeled images, | 1360 | Images, text | Classification | 2006 | [311][312] | M-E Nilsback et al. |
Plant Seedlings Dataset | 12 category dataset of plant seedlings. | Labelled images, segmented images, | 5544 | Images | Classification, detection | 2017 | [313] | Giselsson et al. |
Fruits-360 | Database with images of 131 fruits and vegetables. | 100x100 pixels, white background. | 90483 | Images (jpg) | Classification | 2017–2024 | [314] | Mihai Oltean |
Microbe
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Ecoli Dataset | Protein localization sites. | Various features of the protein localizations sites are given. | 336 | Text | Classification | 1996 | [315][316] | K. Nakai et al. |
MicroMass Dataset | Identification of microorganisms from mass-spectrometry data. | Various mass spectrometer features. | 931 | Text | Classification | 2013 | [317][318] | P. Mahe et al. |
Yeast Dataset | Predictions of Cellular localization sites of proteins. | Eight features given per instance. | 1484 | Text | Classification | 1996 | [319][320] | K. Nakai et al. |
Drug discovery
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Tox21 Dataset | Prediction of outcome of biological assays. | Chemical descriptors of molecules are given. | 12707 | Text | Classification | 2016 | [321] | A. Mayr et al. |
Anomaly data
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Numenta Anomaly Benchmark (NAB) | Data are ordered, timestamped, single-valued metrics. All data files contain anomalies, unless otherwise noted. | None | 50+ files | CSV | Anomaly detection | 2016 (continually updated) | [322] | Numenta |
Skoltech Anomaly Benchmark (SKAB) | Each file represents a single experiment and contains a single anomaly. The dataset represents a multivariate time series collected from the sensors installed on the testbed. | There are two markups for Outlier detection (point anomalies) and Changepoint detection (collective anomalies) problems | 30+ files (v0.9) | CSV | Anomaly detection | 2020 (continually updated) | Iurii D. Katser and Vyacheslav O. Kozitsin | |
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study | Most data files are adapted from UCI Machine Learning Repository data, some are collected from the literature. | treated for missing values, numerical attributes only, different percentages of anomalies, labels | 1000+ files | ARFF | Anomaly detection | 2016 (possibly updated with new datasets and/or results) | Campos et al. |
Question answering data
[edit]This section includes datasets that deals with structured data.
Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
DBpedia Neural Question Answering (DBNQA) Dataset | A large collection of Question to SPARQL specially design for Open Domain Neural Question Answering over DBpedia Knowledgebase. | This dataset contains a large collection of Open Neural SPARQL Templates and instances for training Neural SPARQL Machines; it was pre-processed by semi-automatic annotation tools as well as by three SPARQL experts. | 894,499 | Question-query pairs | Question Answering | 2018 | [326][327] | Hartmann, Soru, and Marx et al. |
Vietnamese Question Answering Dataset (UIT-ViQuAD) | A large collection of Vietnamese questions for evaluating MRC models. | This dataset comprises over 23,000 human-generated question-answer pairs based on 5,109 passages of 174 Vietnamese articles from Wikipedia. | 23,074 | Question-answer pairs | Question Answering | 2020 | [328] | Nguyen et al. |
Vietnamese Multiple-Choice Machine Reading Comprehension Corpus(ViMMRC) | A collection of Vietnamese multiple-choice questions for evaluating MRC models. | This corpus includes 2,783 Vietnamese multiple-choice questions. | 2,783 | Question-answer pairs | Question Answering/Machine Reading Comprehension | 2020 | [329] | Nguyen et al. |
Open-Domain Question Answering Goes Conversational via Question Rewriting | An end-to-end open-domain question answering. | This dataset includes 14,000 conversations with 81,000 question-answer pairs. | Context, Question, Rewrite, Answer, Answer_URL, Conversation_no, Turn_no, Conversation_source Further details are provided in the project's GitHub repository and respective Hugging Face dataset card. | Question Answering | 2021 | [330] | Anantha and Vakulenko et al. | |
UnifiedQA | Question-answer data | Processed dataset | Question Answering | 2020 | [331] | Khashabi et al. |
Dialog or instruction prompted data
[edit]This section includes datasets that ...
Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Taskmaster | "The Taskmaster corpus consists of THREE datasets, Taskmaster-1 (TM-1), Taskmaster-2 (TM-2), and Taskmaster-3 (TM-3), comprising over 55,000 spoken and written task-oriented dialogs in over a dozen domains."[332] | Taskmaster-1: goal-oriented conversational dataset. It includes 13,215 task-based dialogs comprising six domains. Taskmaster-2: 17,289 dialogs in the seven domains (restaurants, food ordering, movies, hotels, flights, music and sports). Taskmaster-3: 23,757 movie ticketing dialogs. | Taskmaster-1 and Taskmaster-2: conversation id, utterances, Instruction id Taskmaster-3: conversation id, utterances, vertical, scenario, instructions. For further details check the project's GitHub repository or the Hugging Face dataset cards (taskmaster-1, taskmaster-2, taskmaster-3). | Dialog/Instruction prompted | 2019 | [333] | Byrne and Krishnamoorthi et al. | |
DrRepair | A labeled dataset for program repair. | Pre-processed data | Check format details in the project's worksheet. | Dialog/Instruction prompted | 2020 | [334] | Michihiro et al. | |
Natural Instructions v2 | Large dataset that covers a wider range of reasoning abilities | Each task consists of input/output, and a task definition. Additionally, each ask contains a task definition. Further information is provided in the GitHub repository of the project and the Hugging Face data card. | Input/Output and task definition | 2022 | [335] | Wang et al. | ||
LAMBADA | " LAMBADA is a collection of narrative passages sharing the characteristic that human subjects are able to guess their last word if they are exposed to the whole passage, but not if they only see the last sentence preceding the target word."[336] | Information about this dataset's format is available in the HuggingFace dataset card and the project's website. The dataset can be downloaded here, and the rejected data here. | 2016 | [337] | Paperno et al. | |||
FLAN | A re-preprocessed version of the FLAN dataset with updates since the original FLAN dataset was released is available in Hugging Face: The scripts to process the data are available in the GitHub repo mentioned on the paper: https://github.com/google-research/FLAN/tree/main/flan. Another FLAN GitHub repo was created as well. This is the one associated with the dataset card in Hugging Face. | 2021 | [338] | Wei et al. |
Cybersecurity
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
MITRE ATTACK | The ATT&CK is a globally-accessible knowledge base of adversary tactics and techniques. | Data can be downloaded from these two GitHub repositories: version 2.1 and version 2.0 | [339] | MITRE ATTACK | ||||
CAPEC | Common Attack Pattern Enumeration and Classification | Data can be downloaded from CAPEC's website: | [340] | CAPEC | ||||
CVE | CVE is a list of publicly disclosed cybersecurity vulnerabilities that is free to search, use, and incorporate into products and services. | Data can be downloaded from: Allitems | [341] | CVE | ||||
CWE | Common Weakness Enumeration data. | Data can be downloaded from: Software Development Hardware Design[permanent dead link]Research Concepts | [342] | CWE | ||||
MalwareTextDB | Annotated database of malware texts. | The GitHub repository of the project contains the data to download. | [343] | Kiat et al. | ||||
USENIX Security Symposium proceedings | Collection of security proceedings from USENIX Security Symposium – technical sessions from 1995 to 2022. | This data is not pre-processed. | 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022. | [344] | USENIX Security Symposium | |||
APTNotes | Collection of public documents, whitepapers and articles about APT campaigns. All the documents are publicly available data. | This data is not pre-processed. | The GitHub repository of the project contains a file with links to the data stored in box. Data files can also be downloaded here. | [345] | APT Notes | |||
arXiv Cryptography and Security papers | Collection of articles about cybersecurity | This data is not pre-processed. | All articles available here. | [346] | arXiv | |||
Security eBooks for free | Small collection of security eBooks, and security presentations publicly available. | This data is not pre-processed. | [347][348][349][350][351][352][353][354][355][356][357][358] | |||||
National Cyber Security strategy repository | Repository of worldwide strategy documents about cybersecurity. | This data is not pre-processed. | [359] | |||||
Cyber Security Natural Language Processing | Data about cybersecurity strategies from more than 75 countries. | Tokenization, meaningless-frequent words removal. | [360] | Yanlin Chen, Yunjian Wei, Yifan Yu, Wen Xue, Xianya Qin | ||||
APT Reports collection | Sample of APT reports, malware, technology, and intelligence collection | Raw and tokenize data available. | All data is available in this GitHub repository. | [citation needed] | blackorbird | |||
Offensive Language Identification Dataset (OLID) | Data available in the project's website. Data is also available here. | [361] | Zampieri et al. | |||||
Cyber reports from the National Cyber Security Centre | This data is not pre-processed. | Threat reports, reports and advisory, news, blog-posts, speeches. | [362] | |||||
APT reports by Kaspersky | This data is not pre-processed. | [363] | ||||||
The cyberwire | This data is not pre-processed. | Newsletters, podcasts, and stories. | [364] | |||||
Databreaches news | This data is not pre-processed. | News, list of news from Aug 2022 to Feb 2023 | [365] | |||||
Cybernews | This data is not pre-processed. | News, curated list of news | [366] | |||||
Bleepingcomputer | This data is not pre-processed. | News | [367] | |||||
Therecord | This data is not pre-processed. | Cybercrime news | [368] | |||||
Hackread | This data is not pre-processed. | Hacking news | [369] | |||||
Securelist | This data is not pre-processed. | APT reports, archive, DDOS reports, incidents, Kaspersky security bulletin, industrial threats, malware-reports, opinions, publications, research, and SAS. | [370] | |||||
Stucco project | The Stucco project collects data not typically integrated into security systems. | This data is not pre-processed | Project's website with data informationReviewed source with links to data sources | [371] | ||||
Farsightsecurity | Website with technical information, reports, and more about security topics. | This data is not pre-processed | Technical information, research, reports. | [372] | ||||
Schneier | Website with academic papers about security topics. | This data is not pre-processed | Papers per category, papers archive by date. | [373] | ||||
Trendmicro | Website with research, news, and perspectives bout security topics. | This data is not pre-processed | Reviewed list of Trendmicro research, news, and perspectives. | [374] | ||||
The Hacker News | News about cybersecurity topics. | This data is not pre-processed | data breaches, cyberattacks, vulnerabilities, malware news. | [375] | ||||
Krebsonsecurity | Security news and investigation | This data is not pre-processed | curated list of news | [376] | ||||
Mitre Defend | Matrix of Defend artifacts | json files | [377] | |||||
Mitre Atlas | Mitre Atlas is a knowledge base of adversary tactics, techniques, and case studies for machine learning (ML) systems based on real-world observations. | This data is not pre-processed | [378] | |||||
Mitre Engage | MITRE Engage is a framework for planning and discussing adversary engagement operations that empowers you to engage your adversaries and achieve your cybersecurity goals. | This data is not pre-processed | [379] | |||||
Hacking Tutorials | This data is not pre-processed | [380] |
Climate and sustainability
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
TCFD reports | Database of company reports that include TCFD-related disclosures. | This data is not pre-processed | Direct link to reportsCurated list of reports | [381] | TCFD Knowledge Hub | |||
Corporate Social Responsibility Reports | A listing of responsibility reports on the internet. | This data is not pre-processed | Curated list of reports | [382] | ResponsibilityReports | |||
The Intergovernmental Panel on Climate Change (IPCC) | A collection of comprehensive assessment reports about knowledge on climate change, its causes, potential impacts and response options | This data is not pre-processed | ReportsCurated list of reports | [383] | IPCC | |||
Alliance for Research on Corporate Sustainability | This data is not pre-processed | Curated list of blog posts | [384] | ARCS | ||||
ESG corpus: Knowledge Hub of the Accounting for Sustainability | This data is not pre-processed | Guides, case studies, blogs, and reports & surveys. | [385] | Mehra et al. | ||||
CLIMATE-FEVER | A dataset adopting the FEVER methodology that consists of 1,535 real-world claims regarding climate-change collected on the internet. | Each claim is accompanied by five manually annotated evidence sentences retrieved from the English Wikipedia that support, refute or do not give enough information to validate the claim totalling in 7,675 claim-evidence pairs.[386] | Dataset HF card, and project's GitHub repository. | [387] | Diggelmann et al. | |||
Climate News dataset | A dataset for NLP and climate change media researchers | The dataset is made up of a number of data artifacts (JSON, JSONL & CSV text files & SQLite database) | Climate news DB, Project's GitHub repository | [388] | ADGEfficiency | |||
Climatext | Climatext is a dataset for sentence-based climate change topic detection. | HF dataset | [389] | University of Zurich | ||||
GreenBiz | Collection of articles and news about climate and sustainability | This data is not pre-processed | Curated list of climate articlesCurated list of sustainability articles | [390] | ||||
Top research pre-prints in climate and sustainability | List of pre-prints from researchers in the reuters hot list | This data is not pre-processed | Curated list of pre-prints | [391] | Maurice Tamman | |||
ARCS | This data is not pre-processed | Curated list of corporate sustainability blogs | [392] | |||||
GreenBiz | Website with articles about climate and sustainability | This data is not pre-processed | [393] | GreenBiz | ||||
CSRWIRE | This data is not pre-processed | Curated list of articles | [394] | CSRWIRE | ||||
CDP | Articles about climate, water, and forests | This data is not pre-processed | [395] | CDP |
Code data
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
The Stack | A 3.1 TB dataset consisting of permissively licensed source code in 30 programming languages. | Filtered through license detection and deduplication. | 6 TB, 51.76B files (prior to deduplication); 3 TB, 5.28B files (after). 358 programming languages. | Parquet | Language modeling, autocompletion, program synthesis. | 2022 | [396][397] | D. Kocetkov, R. Li, L. Ben Allal, L. von Werra, H. de Vries |
GitHub repositories | This data is not pre-processed | Curated lis of repositories from GitHub: 61 62 63 64 65 66 67 68 69 70 71, 72, 73, 74, 75, 76, 77 101 | ||||||
IBM Public GitHub repositories | This data is not pre-processed | Curated list of repositories from GitHub | ||||||
RedHat Public GitHub repositories | This data is not pre-processed | Curated list of repositories from GitHub | ||||||
StackExchange Public Archive.org files | This data is not pre-processed | Curated list of files from Archive.org | ||||||
Gitlab Public repositories | This data is not pre-processed | Curated list of repositories from Gitlab: 1 2 | ||||||
Ansible Collections public repositories | This data is not pre-processed | Curated list of repositories from GitHub. | ||||||
CodeParrot GitHub Code Dataset | This data is not pre-processed | Curated list of repositories from Hugging Face: 1 2 3 4 5 6 7 8 9 10 | ||||||
OKD | The Community Distribution of Kubernetes that powers Red Hat OpenShift | This data is not pre-processed | List of GitHub repositories of the project | |||||
OpenShift | The developer and operations friendly Kubernetes distro | List of GitHub repositories of the project | ||||||
Kubernetes | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Red Hat Developer | GitHub home of the Red Hat Developer program | This data is not pre-processed | List of GitHub repositories of the project | |||||
Red Hat Workshops | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Kubernetes SIGs | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Konveyor | This data is not pre-processed | List of GitHub repositories of the project | ||||||
RedHat Marketplace | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Redhat blog | This data is not pre-processed | [398] | ||||||
Kubernetes io | This data is not pre-processed | [399] | ||||||
Docs Openshift | This data is not pre-processed | [400] | ||||||
cncf io | This data is not pre-processed | [401] | ||||||
Kubernetes presentations | List of publicly available Kubernetes presentations | This data is not pre-processed | data link | |||||
Red Hat Open Innovation Labs | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Red Hat Demos | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Red Hat OpenShift Online | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Software Collections | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Red Hat Insights | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Red Hat Government | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Red Hat Consulting | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Red Hat Communities of Practice | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Red Hat Partner Tech | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Red Hat Documentation | This data is not pre-processed | List of GitHub repositories of the project | ||||||
IBM | This data is not pre-processed | List of GitHub repositories of the project | ||||||
IBM Cloud | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Build Lab Team | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Terraform IBM Modules | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Cloud Schematics | This data is not pre-processed | List of GitHub repositories of the project | ||||||
OCP Power Demos | This data is not pre-processed | List of GitHub repositories of the project | ||||||
IBM App Modernization | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Kubernetes OperatorHub | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Cloud Native Computing Foundation (CNCF) | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Operator Framework | This data is not pre-processed | List of GitHub repositories of the project | [402] | |||||
GitHub repositories referenced in artifacthub.io | This data is not pre-processed | List of GitHub repositories in artifacthub.io | ||||||
Red Hat Communities of Practice | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Red Hat partner | This data is not pre-processed | List of GitHub repositories of the project | ||||||
IBM Repositories | This data is not pre-processed | List of GitHub repositories for the project | ||||||
Build Lab Team | This data is not pre-processed | List of GitHub repositories for the project | ||||||
Operator Framework | This data is not pre-processed | List of GitHub repositories for the project | ||||||
GitHub repositories | This data is not pre-processed | List of GitHub repositories for the project | ||||||
Red Hat | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Kubernetes Patterns | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Kubernetes Deployment & Security Patterns | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Kubernetes for Full-Stack Developers | This data is not pre-processed | List of GitHub repositories of the project | ||||||
Load Balancer Cloudwatch Metrics | This data is not pre-processed | GitHub repository of the project | ||||||
Dynatrace | This data is not pre-processed | [5] | ||||||
AIOps Challenge 2020 Data | This data is not pre-processed | GitHub repository of the project | ||||||
Loghub | This data is not pre-processed | List of repositories | ||||||
HTML Pages | This data is not pre-processed | List of HTML pages | ||||||
Opensift ebooks | This data is not pre-processed | [403] | ||||||
Kubernetes ebooks | This data is not pre-processed | Kubernetes Patterns, Kubernetes Deployment, Kubernetes for Full-Stack Developers | ||||||
Kubernetes for Full-Stack Developers | This data is not pre-processed | Kubernetes for Full-Stack Developers | ||||||
List of public and licensed Github repositories | This data is not pre-processed | List of repositories |
Multivariate data
[edit]Financial
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Dow Jones Index | Weekly data of stocks from the first and second quarters of 2011. | Calculated values included such as percentage change and a lags. | 750 | Comma separated values | Classification, regression, Time series | 2014 | [404][405] | M. Brown et al. |
Statlog (Australian Credit Approval) | Credit card applications either accepted or rejected and attributes about the application. | Attribute names are removed as well as identifying information. Factors have been relabeled. | 690 | Comma separated values | Classification | 1987 | [406][407] | R. Quinlan |
eBay auction data | Auction data from various eBay.com objects over various length auctions | Contains all bids, bidderID, bid times, and opening prices. | ~ 550 | Text | Regression, classification | 2012 | [408][409] | G. Shmueli et al. |
Statlog (German Credit Data) | Binary credit classification into "good" or "bad" with many features | Various financial features of each person are given. | 690 | Text | Classification | 1994 | [410] | H. Hofmann |
Bank Marketing Dataset | Data from a large marketing campaign carried out by a large bank . | Many attributes of the clients contacted are given. If the client subscribed to the bank is also given. | 45,211 | Text | Classification | 2012 | [411][412] | S. Moro et al. |
Istanbul Stock Exchange Dataset | Several stock indexes tracked for almost two years. | None. | 536 | Text | Classification, regression | 2013 | [413][414] | O. Akbilgic |
Default of Credit Card Clients | Credit default data for Taiwanese creditors. | Various features about each account are given. | 30,000 | Text | Classification | 2016 | [415][416] | I. Yeh |
StockNet | Stock movement prediction from tweets and historical stock prices | None | Text | NLP | 2018 | [417] | Yumo Xu and Shay B. Cohen |
Weather
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Cloud DataSet | Data about 1024 different clouds. | Image features extracted. | 1024 | Text | Classification, clustering | 1989 | [418] | P. Collard |
El Nino Dataset | Oceanographic and surface meteorological readings taken from a series of buoys positioned throughout the equatorial Pacific. | 12 weather attributes are measured at each buoy. | 178080 | Text | Regression | 1999 | [419] | Pacific Marine Environmental Laboratory |
Greenhouse Gas Observing Network Dataset | Time-series of greenhouse gas concentrations at 2921 grid cells in California created using simulations of the weather. | None. | 2921 | Text | Regression | 2015 | [420] | D. Lucas |
Atmospheric CO2 from Continuous Air Samples at Mauna Loa Observatory | Continuous air samples in Hawaii, USA. 44 years of records. | None. | 44 years | Text | Regression | 2001 | [421] | Mauna Loa Observatory |
Ionosphere Dataset | Radar data from the ionosphere. Task is to classify into good and bad radar returns. | Many radar features given. | 351 | Text | Classification | 1989 | [279][422] | Johns Hopkins University |
Ozone Level Detection Dataset | Two ground ozone level datasets. | Many features given, including weather conditions at time of measurement. | 2536 | Text | Classification | 2008 | [423][424] | K. Zhang et al. |
Census
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Adult Dataset | Census data from 1994 containing demographic features of adults and their income. | Cleaned and anonymized. | 48,842 | Comma separated values | Classification | 1996 | [425] | United States Census Bureau |
Census-Income (KDD) | Weighted census data from the 1994 and 1995 Current Population Surveys. | Split into training and test sets. | 299,285 | Comma separated values | Classification | 2000 | [426][427] | United States Census Bureau |
IPUMS Census Database | Census data from the Los Angeles and Long Beach areas. | None | 256,932 | Text | Classification, regression | 1999 | [428] | IPUMS |
US Census Data 1990 | Partial data from 1990 US census. | Results randomized and useful attributes selected. | 2,458,285 | Text | Classification, regression | 1990 | [429] | United States Census Bureau |
Transit
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Bike Sharing Dataset | Hourly and daily count of rental bikes in a large city. | Many features, including weather, length of trip, etc., are given. | 17,389 | Text | Regression | 2013 | [430][431] | H. Fanaee-T |
New York City Taxi Trip Data | Trip data for yellow and green taxis in New York City. | Gives pick up and drop off locations, fares, and other details of trips. | 6 years | Text | Classification, clustering | 2015 | [432] | New York City Taxi and Limousine Commission |
Taxi Service Trajectory ECML PKDD | Trajectories of all taxis in a large city. | Many features given, including start and stop points. | 1,710,671 | Text | Clustering, causal-discovery | 2015 | [433][434] | M. Ferreira et al. |
METR-LA | Speed from loop detectors in the highway of Los Angeles County. | Average speed in 5 minutes timesteps. | 7,094,304 from 207 sensors and 34,272 timesteps | Comma separated values | Regression, Forecasting | 2014 | [435] | Jagadish et al. |
PeMS | Speed, flow, occupancy and other metrics from loop detectors and other sensors in the freeway of the State of California, U.S.A.. | Metric usually aggregated via Average into 5 minutes timesteps. | 39,000 individual detectors, each containing years of timeseries | Comma separated values | Regression, Forecasting, Nowcasting, Interpolation | (updated realtime) | [436] | California Department of Transportation |
Internet
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Webpages from Common Crawl 2012 | Large collection of webpages and how they are connected via hyperlinks | None. | 3.5B | Text | clustering, classification | 2013 | [437] | V. Granville |
Internet Advertisements Dataset | Dataset for predicting if a given image is an advertisement or not. | Features encode geometry of ads and phrases occurring in the URL. | 3279 | Text | Classification | 1998 | [438][439] | N. Kushmerick |
Internet Usage Dataset | General demographics of internet users. | None. | 10,104 | Text | Classification, clustering | 1999 | [440] | D. Cook |
URL Dataset | 120 days of URL data from a large conference. | Many features of each URL are given. | 2,396,130 | Text | Classification | 2009 | [441][442] | J. Ma |
Phishing Websites Dataset | Dataset of phishing websites. | Many features of each site are given. | 2456 | Text | Classification | 2015 | [443] | R. Mustafa et al. |
Online Retail Dataset | Online transactions for a UK online retailer. | Details of each transaction given. | 541,909 | Text | Classification, clustering | 2015 | [444] | D. Chen |
Freebase Simple Topic Dump | Freebase is an online effort to structure all human knowledge. | Topics from Freebase have been extracted. | large | Text | Classification, clustering | 2011 | [445][446] | Freebase |
Farm Ads Dataset | The text of farm ads from websites. Binary approval or disapproval by content owners is given. | SVMlight sparse vectors of text words in ads calculated. | 4143 | Text | Classification | 2011 | [447][448] | C. Masterharm et al. |
The Pile | Assembling several large datasets of diverse and unstructured texts | Various (removing HTML and Javascript from websites, removing duplicated sentences) | 825 GiB English text | JSON Lines[449][450] | Natural Language Processing, Text Prediction | 2021 | [451][449] | Gao et al. |
OSCAR | Large collection of monolingual corpora extracted from web data (Common Crawl dumps) covering 150+ languages | Various (filtering, language classification, adult-content detection and other labelling) | 3.4 TB English text, 1.4 TB Chinese text, 1.1 TB Russian text, 595 MB German text, 431 MB French text, and data for 150+ languages (figures for version 23.01) | JSON Lines[452] | Natural Language Processing, Text Prediction | 2021 | [453][454] | Ortiz Suarez, Abadji, Sagot et al. |
OpenWebText | An open-source recreation of the WebText corpus. The text is web content extracted from URLs shared on Reddit with at least three upvotes. | Extracted non-HTML content, deduplicated, and tokenized. | 8,013,769 Documents, 38GB | Text | Natural Language Processing, Text Prediction | 2019 | [455][456] | A. Gokaslan, V. Cohen |
ROOTS | A well-documented and representative multilingual dataset with the explicit goal of doing good for and by the people whose data was collected. | Extracted non-HTML content, cleaned out UI and ads, deduplicated, removed PII, and tokenized. | 1.6 TB, 59 languages. | Parquet | Natural Language Processing, Text Prediction | 2022 | [457][458] | H. Laurençon, L. Saulnier, T. Wang, C. Akiki, A. Villanova del Moral, T. Le Scao |
Games
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Poker Hand Dataset | 5 card hands from a standard 52 card deck. | Attributes of each hand are given, including the Poker hands formed by the cards it contains. | 1,025,010 | Text | Regression, classification | 2007 | [459] | R. Cattral |
Connect-4 Dataset | Contains all legal 8-ply positions in the game of connect-4 in which neither player has won yet, and in which the next move is not forced. | None. | 67,557 | Text | Classification | 1995 | [460] | J. Tromp |
Chess (King-Rook vs. King) Dataset | Endgame Database for White King and Rook against Black King. | None. | 28,056 | Text | Classification | 1994 | [461][462] | M. Bain et al. |
Chess (King-Rook vs. King-Pawn) Dataset | King+Rook versus King+Pawn on a7. | None. | 3196 | Text | Classification | 1989 | [463] | R. Holte |
Tic-Tac-Toe Endgame Dataset | Binary classification for win conditions in tic-tac-toe. | None. | 958 | Text | Classification | 1991 | [464] | D. Aha |
Other multivariate
[edit]Dataset Name | Brief description | Preprocessing | Instances | Format | Default Task | Created (updated) | Reference | Creator |
---|---|---|---|---|---|---|---|---|
Housing Data Set | Median home values of Boston with associated home and neighborhood attributes. | None. | 506 | Text | Regression | 1993 | [465] | D. Harrison et al. |
The Getty Vocabularies | structured terminology for art and other material culture, archival materials, visual surrogates, and bibliographic materials. | None. | large | Text | Classification | 2015 | [466] | Getty Center |
Yahoo! Front Page Today Module User Click Log | User click log for news articles displayed in the Featured Tab of the Today Module on Yahoo! Front Page. | Conjoint analysis with a bilinear model. | 45,811,883 user visits | Text | Regression, clustering | 2009 | [467][468] | Chu et al. |
British Oceanographic Data Centre | Biological, chemical, physical and geophysical data for oceans. 22K variables tracked. | Various. | 22K variables, many instances | Text | Regression, clustering | 2015 | [469] | British Oceanographic Data Centre |
Congressional Voting Records Dataset | Voting data for all USA representatives on 16 issues. | Beyond the raw voting data, various other features are provided. | 435 | Text | Classification | 1987 | [470] | J. Schlimmer |
Entree Chicago Recommendation Dataset | Record of user interactions with Entree Chicago recommendation system. | Details of each users usage of the app are recorded in detail. | 50,672 | Text | Regression, recommendation | 2000 | [471] | R. Burke |
Insurance Company Benchmark (COIL 2000) | Information on customers of an insurance company. | Many features of each customer and the services they use. | 9,000 | Text | Regression, classification | 2000 | [472][473] | P. van der Putten |
Nursery Dataset | Data from applicants to nursery schools. | Data about applicant's family and various other factors included. | 12,960 | Text | Classification | 1997 | [474][475] | V. Rajkovic et al. |
University Dataset | Data describing attributed of a large number of universities. | None. | 285 | Text | Clustering, classification | 1988 | [476] | S. Sounders et al. |
Blood Transfusion Service Center Dataset | Data from blood transfusion service center. Gives data on donors return rate, frequency, etc. | None. | 748 | Text | Classification | 2008 | [477][478] | I. Yeh |
Record Linkage Comparison Patterns Dataset | Large dataset of records. Task is to link relevant records together. | Blocking procedure applied to select only certain record pairs. | 5,749,132 | Text | Classification | 2011 | [479][480] | University of Mainz |
Nomao Dataset | Nomao collects data about places from many different sources. Task is to detect items that describe the same place. | Duplicates labeled. | 34,465 | Text | Classification | 2012 | [481][482] | Nomao Labs |
Movie Dataset | Data for 10,000 movies. | Several features for each movie are given. | 10,000 | Text | Clustering, classification | 1999 | [483] | G. Wiederhold |
Open University Learning Analytics Dataset | Information about students and their interactions with a virtual learning environment. | None. | ~ 30,000 | Text | Classification, clustering, regression | 2015 | [484][485] | J. Kuzilek et al. |
Mobile phone records | Telecommunications activity and interactions | Aggregation per geographical grid cells and every 15 minutes. | large | Text | Classification, Clustering, Regression | 2015 | [486] | G. Barlacchi et al. |
Curated repositories of datasets
[edit]As datasets come in myriad formats and can sometimes be difficult to use, there has been considerable work put into curating and standardizing the format of datasets to make them easier to use for machine learning research.
- OpenML:[487] Web platform with Python, R, Java, and other APIs for downloading hundreds of machine learning datasets, evaluating algorithms on datasets, and benchmarking algorithm performance against dozens of other algorithms.
- PMLB:[488] A large, curated repository of benchmark datasets for evaluating supervised machine learning algorithms. Provides classification and regression datasets in a standardized format that are accessible through a Python API.
- Metatext NLP: https://metatext.io/datasets web repository maintained by community, containing nearly 1000 benchmark datasets, and counting. Provides many tasks from classification to QA, and various languages from English, Portuguese to Arabic.
- Appen: Off The Shelf and Open Source Datasets hosted and maintained by the company. These biological, image, physical, question answering, signal, sound, text, and video resources number over 250 and can be applied to over 25 different use cases.[489][490]
See also
[edit]- Comparison of deep learning software
- List of manual image annotation tools
- List of biological databases
References
[edit]- ^ Wissner-Gross, A. "Datasets Over Algorithms". Edge.com. Retrieved 8 January 2016.
- ^ Weiss, G. M.; Provost, F. (1 September 2003). "Learning When Training Data are Costly: The Effect of Class Distribution on Tree Induction". Journal of Artificial Intelligence Research. 19. AI Access Foundation: 315–354. doi:10.1613/jair.1199. ISSN 1076-9757. S2CID 2344521.
- ^ Turney, Peter (2000). "Types of cost in inductive concept learning". arXiv:cs/0212034.
- ^ Abney, Steven (17 September 2007). Semisupervised Learning for Computational Linguistics. CRC Press. ISBN 978-1-4200-1080-0.
- ^ Žliobaitė, Indrė; Bifet, Albert; Pfahringer, Bernhard; Holmes, Geoff (2011). "Active Learning with Evolving Streaming Data". Machine Learning and Knowledge Discovery in Databases. Lecture Notes in Computer Science. Vol. 6913. Berlin, Heidelberg: Springer Berlin Heidelberg. pp. 597–612. doi:10.1007/978-3-642-23808-6_39. ISBN 978-3-642-23807-9. ISSN 0302-9743.
- ^ McAuley, Julian; Targett, Christopher; Shi, Qinfeng; Anton van den Hengel (2015). "Image-based Recommendations on Styles and Substitutes". arXiv:1506.04757 [cs.CV].
- ^ "Amazon review data". nijianmo.github.io. Retrieved 8 October 2021.
- ^ Ganesan, Kavita; Zhai, Chengxiang (2012). "Opinion-based entity ranking". Information Retrieval. 15 (2): 116–150. doi:10.1007/s10791-011-9174-8. hdl:2142/15252. S2CID 16258727.
- ^ Lv, Yuanhua, Dimitrios Lymberopoulos, and Qiang Wu. "An exploration of ranking heuristics in mobile local search." Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval. ACM, 2012.
- ^ Harper, F. Maxwell; Konstan, Joseph A. (2015). "The MovieLens Datasets: History and Context". ACM Transactions on Interactive Intelligent Systems. 5 (4): 19. doi:10.1145/2827872. S2CID 16619709.
- ^ Koenigstein, Noam, Gideon Dror, and Yehuda Koren. "Yahoo! music recommendations: modeling music ratings with temporal dynamics and item taxonomy." Proceedings of the fifth ACM conference on Recommender systems. ACM, 2011.
- ^ McFee, Brian, et al. "The million song dataset challenge." Proceedings of the 21st international conference companion on World Wide Web. ACM, 2012.
- ^ Bohanec, Marko, and Vladislav Rajkovic. "Knowledge acquisition and explanation for multi-attribute decision making." 8th Intl Workshop on Expert Systems and their Applications. 1988.
- ^ Tan, Peter J., and David L. Dowe. "MML inference of decision graphs with multi-way joins." Australian Joint Conference on Artificial Intelligence. 2002.
- ^ "Quantifying comedy on YouTube: why the number of o's in your LOL matter". Metatext NLP Database. Retrieved 26 October 2020.
- ^ Kim, Byung Joo (2012). "A Classifier for Big Data". Convergence and Hybrid Information Technology. Communications in Computer and Information Science. Vol. 310. pp. 505–512. doi:10.1007/978-3-642-32692-9_63. ISBN 978-3-642-32691-2.
- ^ Pérezgonzález, Jose D.; Gilbey, Andrew (2011). "Predicting Skytrax airport rankings from customer reviews". Journal of Airport Management. 5 (4): 335–339. doi:10.69554/RFZC4321.
- ^ Loh, Wei-Yin, and Yu-Shan Shih. "Split selection methods for classification trees." Statistica sinica(1997): 815–840.
- ^ Lim, Tjen-Sien; Loh, Wei-Yin; Shih, Yu-Shan (2000). "A comparison of prediction accuracy, complexity, and training time of thirty-three old and new classification algorithms". Machine Learning. 40 (3): 203–228. doi:10.1023/a:1007608224229. S2CID 17030953.
- ^ Kiet Van Nguyen, Vu Duc Nguyen, Phu X. V. Nguyen, Tham T. H. Truong, Ngan Luu-Thuy Nguyen. "UIT-VSFC: Vietnamese Students’ Feedback Corpus for Sentiment Analysis
- ^ Ho, Vong Anh; Nguyen, Duong Huynh-Cong; Nguyen, Danh Hoang; Pham, Linh Thi-Van; Nguyen, Duc-Vu; Nguyen, Kiet Van; Nguyen, Ngan Luu-Thuy (2020). "Emotion Recognition for Vietnamese Social Media Text". Computational Linguistics. Communications in Computer and Information Science. Vol. 1215. pp. 319–333. arXiv:1911.09339. doi:10.1007/978-981-15-6168-9_27. ISBN 978-981-15-6167-2. S2CID 208202333.
- ^ Nhung Thi-Hong Nguyen, Phuong Ha-Dieu Phan, Luan Thanh Nguyen, Kiet Van Nguyen, Ngan Luu-Thuy Nguyen (24 April 2021). "Vietnamese Open-domain Complaint Detection in E-Commerce Websites". arXiv:2104.11969 [cs.CL].
{{cite arXiv}}
: CS1 maint: multiple names: authors list (link) - ^ Phu Gia Hoang, Canh Duc Luu, Khanh Quoc Tran, Kiet Van Nguyen, Ngan Luu-Thuy Nguyen (26 January 2023). "ViHOS: Hate Speech Spans Detection for Vietnamese". arXiv:2301.10186 [cs.CL].
{{cite arXiv}}
: CS1 maint: multiple names: authors list (link) - ^ Dermouche, Mohamed; Velcin, Julien; Khouas, Leila; Loudcher, Sabine (2014). "A Joint Model for Topic-Sentiment Evolution over Time". 2014 IEEE International Conference on Data Mining. IEEE. pp. 773–778. doi:10.1109/icdm.2014.82. ISBN 978-1-4799-4302-9.
- ^ Rose, Tony; Stevenson, Mark; Whitehead, Miles (2002). "The Reuters Corpus Volume 1-from Yesterday's News to Tomorrow's Language Resources" (PDF). LREC. 2. S2CID 9239414. Archived from the original (PDF) on 6 August 2019.
- ^ Amini, Massih R.; Usunier, Nicolas; Goutte, Cyril (2009). "Learning from Multiple Partially Observed Views – an Application to Multilingual Text Categorization". Advances in Neural Information Processing Systems. 22: 28–36.
- ^ Liu, Ming; et al. (2015). "VRCA: a clustering algorithm for massive amount of texts". Proceedings of the 24th International Conference on Artificial Intelligence. AAAI Press. Archived from the original on 5 November 2021. Retrieved 6 August 2019.
- ^ Al-Harbi, S; Almuhareb, A; Al-Thubaity, A; Khorsheed, M. S.; Al-Rajeh, A (2008). "Automatic Arabic Text Classification". Proceedings of the 9th International Conference on the Statistical Analysis of Textual Data, Lyon, France.
- ^ "Relationship and Entity Extraction Evaluation Dataset: Dstl/re3d". GitHub. 17 December 2018.
- ^ "The Examiner – SpamClickBait Catalogue".
- ^ "A Million News Headlines".
- ^ "One Week of Global News Feeds".
- ^ Kulkarni, Rohit (2018), Reuters News-Wire Archive, Harvard Dataverse, doi:10.7910/DVN/XDB74W
- ^ "IrishTimes – the Waxy-Wany News".
- ^ "News Headlines Dataset For Sarcasm Detection". kaggle.com. Retrieved 27 April 2019.
- ^ Klimt, Bryan, and Yiming Yang. "Introducing the Enron Corpus." CEAS. 2004.
- ^ Kossinets, Gueorgi; Kleinberg, Jon; Watts, Duncan (2008). "The Structure of Information Pathways in a Social Communication Network". arXiv:0806.3201 [physics.soc-ph].
- ^ Androutsopoulos, Ion; Koutsias, John; Chandrinos, Konstantinos V.; Paliouras, George; Spyropoulos, Constantine D. (2000). "An evaluation of Naive Bayesian anti-spam filtering". In Potamias, G.; Moustakis, V.; van Someren, M. (eds.). Proceedings of the Workshop on Machine Learning in the New Information Age. 11th European Conference on Machine Learning, Barcelona, Spain. Vol. 11. pp. 9–17. arXiv:cs/0006013. Bibcode:2000cs........6013A.
- ^ Bratko, Andrej; et al. (2006). "Spam filtering using statistical data compression models" (PDF). The Journal of Machine Learning Research. 7: 2673–2698.
- ^ Almeida, Tiago A., José María G. Hidalgo, and Akebo Yamakami. "Contributions to the study of SMS spam filtering: new collection and results."Proceedings of the 11th ACM symposium on Document engineering. ACM, 2011.
- ^ Delany; Jane, Sarah; Buckley, Mark; Greene, Derek (2012). "SMS spam filtering: methods and data". Expert Systems with Applications. 39 (10): 9899–9908. doi:10.1016/j.eswa.2012.02.053. S2CID 15546924.
- ^ Joachims, Thorsten. A Probabilistic Analysis of the Rocchio Algorithm with TFIDF for Text Categorization. No. CMU-CS-96-118. Carnegie-mellon univ pittsburgh pa dept of computer science, 1996.
- ^ Dimitrakakis, Christos, and Samy Bengio. Online Policy Adaptation for Ensemble Algorithms. No. EPFL-REPORT-82788. IDIAP, 2002.
- ^ Dooms, S. et al. "Movietweetings: a movie rating dataset collected from twitter, 2013. Available from https://github.com/sidooms/MovieTweetings."
- ^ RoyChowdhury, Aruni; Lin, Tsung-Yu; Maji, Subhransu; Learned-Miller, Erik (2017). "Twitter100k: A Real-world Dataset for Weakly Supervised Cross-Media Retrieval". arXiv:1703.06618 [cs.CV].
- ^ "huyt16/Twitter100k". GitHub. Retrieved 26 March 2018.
- ^ Go, Alec; Bhayani, Richa; Huang, Lei (2009). "Twitter sentiment classification using distant supervision". CS224N Project Report, Stanford. 1: 12.
- ^ Chikersal, Prerna, Soujanya Poria, and Erik Cambria. "SeNTU: sentiment analysis of tweets by combining a rule-based classifier with supervised learning." Proceedings of the International Workshop on Semantic Evaluation, SemEval. 2015.
- ^ Zafarani, Reza, and Huan Liu. "Social computing data repository at ASU." School of Computing, Informatics and Decision Systems Engineering, Arizona State University (2009).
- ^ Data Science Course by DataTrained Education "IBM Certified Data Science Course." IBM Certified Online Data Science Course
- ^ McAuley, Julian J.; Leskovec, Jure. "Learning to Discover Social Circles in Ego Networks". NIPS. 2012: 2012.
- ^ Šubelj, Lovro; Fiala, Dalibor; Bajec, Marko (2014). "Network-based statistical comparison of citation topology of bibliographic databases". Scientific Reports. 4 (6496): 6496. arXiv:1502.05061. Bibcode:2014NatSR...4E6496S. doi:10.1038/srep06496. PMC 4178292. PMID 25263231.
- ^ Abdulla, N., et al. "Arabic sentiment analysis: Corpus-based and lexicon-based." Proceedings of the IEEE conference on Applied Electrical Engineering and Computing Technologies (AEECT). 2013.
- ^ Abooraig, Raddad, et al. "On the automatic categorization of Arabic articles based on their political orientation." Third International Conference on Informatics Engineering and Information Science (ICIEIS2014). 2014.
- ^ Kawala, François, et al. "Prédictions d'activité dans les réseaux sociaux en ligne." 4ième conférence sur les modèles et l'analyse des réseaux: Approches mathématiques et informatiques. 2013.
- ^ Sabharwal, Ashish; Samulowitz, Horst; Tesauro, Gerald (2015). "Selecting Near-Optimal Learners via Incremental Data Allocation". arXiv:1601.00024 [cs.LG].
- ^ Xu et al. "SemEval-2015 Task 1: Paraphrase and Semantic Similarity in Twitter (PIT)" Proceedings of the 9th International Workshop on Semantic Evaluation. 2015.
- ^ Xu et al. "Extracting Lexically Divergent Paraphrases from Twitter" Transactions of the Association for Computational (TACL). 2014.
- ^ Middleton, Stuart E; Middleton, Lee; Modafferi, Stefano (2014). "Real-Time Crisis Mapping of Natural Disasters Using Social Media" (PDF). IEEE Intelligent Systems. 29 (2): 9–17. doi:10.1109/MIS.2013.126. S2CID 15139204.
- ^ "geoparsepy". 2016. Python PyPI library
- ^ Shmueli, Boaz; Ku, Lun-Wei; Ray, Soumya (2020). "Reactive Supervision: A New Method for Collecting Sarcasm Data". Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics. pp. 2553–2559. doi:10.18653/v1/2020.emnlp-main.201. S2CID 221970454.
- ^ Shmueli, Boaz. "SPIRS Sarcasm Dataset". GitHub.
- ^ Gupta, Aakash (2020). "Dutch social media collection". COVID-19 Data Hub. doi:10.5072/FK2/MTPTL7. Retrieved 11 November 2023.
- ^ "Streamlit". huggingface.co. Retrieved 18 December 2020.
- ^ "Dutch Social media collection". kaggle.com. Retrieved 18 December 2020.
- ^ Shmueli, Boaz; Ray, Soumya; Lun-Wei (2021). "Happy Dance, Slow Clap: Using Reaction GIFs to Predict Induced Affect on Twitter". Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). Vol. Association for Computational Linguistics. As. pp. 395–401. doi:10.18653/v1/2021.acl-short.50. S2CID 235125510.
- ^ Shmueli, Boaz (5 May 2023), ReactionGIF, retrieved 6 October 2023
- ^ Forsyth, E., Lin, J., & Martell, C. (2008, June 25). The NPS Chat Corpus. Retrieved from http://faculty.nps.edu/cmartell/NPSChat.htm
- ^ Sordoni, Alessandro; Galley, Michel; Auli, Michael; Brockett, Chris; Ji, Yangfeng; Mitchell, Margaret; Nie, Jian-Yun; Gao, Jianfeng; Dolan, Bill (2015). "A Neural Network Approach to Context-Sensitive Generation of Conversational Responses". arXiv:1506.06714 [cs.CL].
- ^ Shaoul, C. & Westbury C. (2013) A reduced redundancy USENET corpus (2005–2011) Edmonton, AB: University of Alberta (downloaded from http://www.psych.ualberta.ca/~westburylab/downloads/usenetcorpus.download.html)
- ^ KAN, M. (2011, January). NUS Short Message Service (SMS) Corpus. Retrieved from http://www.comp.nus.edu.sg/entrepreneurship/innovation/osr/corpus/ Archived 29 June 2018 at the Wayback Machine
- ^ Stuck_In_the_Matrix. (2015, July 3). I have every publicly available Reddit comment for research. ~ 1.7 billion comments @ 250 GB compressed. Any interest in this? [Original post]. Message posted to https://www.reddit.com/r/datasets/comments/3bxlg7/i_have_every_publicly_available_reddit_comment/
- ^ Lowe, Ryan; Pow, Nissan; Serban, Iulian; Pineau, Joelle (2015). "The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems". arXiv:1506.08909 [cs.CL].
- ^ Jason Williams Antoine Raux Matthew Henderson, "[1]", Dialogue & Discourse | April 2016 .
- ^ Hoppe, Travis (16 December 2021), The-Pile-FreeLaw, retrieved 11 January 2023
- ^ Zheng, Lucia; Guha, Neel; Anderson, Brandon R.; Henderson, Peter; Ho, Daniel E. (21 June 2021). "When does pretraining help?". Proceedings of the Eighteenth International Conference on Artificial Intelligence and Law. New York, NY, USA: ACM. pp. 159–168. doi:10.1145/3462757.3466088. ISBN 9781450385268. S2CID 233296302.
- ^ "pile-of-law/pile-of-law · Datasets at Hugging Face". huggingface.co. 4 July 2022. Retrieved 11 January 2023.
- ^ "About | Caselaw Access Project". case.law. Retrieved 11 January 2023.
- ^ K. Kowsari, D. E. Brown, M. Heidarysafa, K. Jafari Meimandi, M. S. Gerber and L. E. Barnes, "HDLTex: Hierarchical Deep Learning for Text Classification", 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 364–371. doi:10.1109/ICMLA.2017.0-134
- ^ K. Kowsari, D. E. Brown, M. Heidarysafa, K. Jafari Meimandi, M. S. Gerber and L. E. Barnes, "Web of Science Dataset", doi:10.17632/9rw3vkcfy4.6
- ^ Galgani, Filippo, Paul Compton, and Achim Hoffmann. "Combining different summarization techniques for legal text." Proceedings of the Workshop on Innovative Hybrid Approaches to the Processing of Textual Data. Association for Computational Linguistics, 2012.
- ^ Nagwani, N. K. (2015). "Summarizing large text collection using topic modeling and clustering based on MapReduce framework". Journal of Big Data. 2 (1): 1–18. doi:10.1186/s40537-015-0020-5.
- ^ Schler, Jonathan; et al. (2006). "Effects of Age and Gender on Blogging" (PDF). AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs. 6. Archived from the original (PDF) on 14 November 2020. Retrieved 6 August 2019.
- ^ Anand, Pranav, et al. "Believe Me-We Can Do This! Annotating Persuasive Acts in Blog Text."Computational Models of Natural Argument. 2011.
- ^ Traud, Amanda L., Peter J. Mucha, and Mason A. Porter. "Social structure of Facebook networks." Physica A: Statistical Mechanics and its Applications391.16 (2012): 4165–4180.
- ^ Richard, Emile; Savalle, Pierre-Andre; Vayatis, Nicolas (2012). "Estimation of Simultaneously Sparse and Low Rank Matrices". arXiv:1206.6474 [cs.DS].
- ^ Richardson, Matthew; Burges, Christopher JC; Renshaw, Erin (2013). "MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text". EMNLP. 1.
- ^ Weston, Jason; Bordes, Antoine; Chopra, Sumit; Rush, Alexander M.; Bart van Merriënboer; Joulin, Armand; Mikolov, Tomas (2015). "Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks". arXiv:1502.05698 [cs.AI].
- ^ Marcus, Mitchell P.; Ann Marcinkiewicz, Mary; Santorini, Beatrice (1993). "Building a large annotated corpus of English: The Penn Treebank". Computational Linguistics. 19 (2): 313–330.
- ^ Collins, Michael (2003). "Head-driven statistical models for natural language parsing". Computational Linguistics. 29 (4): 589–637. doi:10.1162/089120103322753356.
- ^ Guyon, Isabelle, et al., eds. Feature extraction: foundations and applications. Vol. 207. Springer, 2008.
- ^ Lin, Yuri, et al. "Syntactic annotations for the google books ngram corpus." Proceedings of the ACL 2012 system demonstrations. Association for Computational Linguistics, 2012.
- ^ Krishnamoorthy, Niveda; et al. (2013). "Generating Natural-Language Video Descriptions Using Text-Mined Knowledge". AAAI. 1. Archived from the original on 6 August 2019. Retrieved 6 August 2019.
- ^ Luyckx, Kim, and Walter Daelemans. "Personae: a Corpus for Author and Personality Prediction from Text[dead link]." LREC. 2008.
- ^ Solorio, Thamar, Ragib Hasan, and Mainul Mizan. "A case study of sockpuppet detection in wikipedia." Workshop on Language Analysis in Social Media (LASM) at NAACL HLT. 2013.
- ^ "Pushshift Files". files.pushshift.io. Archived from the original on 12 January 2023. Retrieved 12 January 2023.
- ^ Baumgartner, Jason; Zannettou, Savvas; Keegan, Brian; Squire, Megan; Blackburn, Jeremy (23 January 2020). "The Pushshift Reddit Dataset". arXiv:2001.08435 [cs.SI].
- ^ Ciarelli, Patrick Marques, and Elias Oliveira. "Agglomeration and elimination of terms for dimensionality reduction." Intelligent Systems Design and Applications, 2009. ISDA'09. Ninth International Conference on. IEEE, 2009.
- ^ Zhou, Mingyuan, Oscar Hernan Madrid Padilla, and James G. Scott. "Priors for random count matrices derived from a family of negative binomial processes." Journal of the American Statistical Association just-accepted (2015): 00–00.
- ^ Kotzias, Dimitrios, et al. "From group to individual labels using deep features." Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2015.
- ^ Ning, Yue; Muthiah, Sathappan; Rangwala, Huzefa; Ramakrishnan, Naren (2016). "Modeling Precursors for Event Forecasting via Nested Multi-Instance Learning". arXiv:1602.08033 [cs.SI].
- ^ Buza, Krisztian. "Feedback prediction for blogs."Data analysis, machine learning and knowledge discovery. Springer International Publishing, 2014. 145–152.
- ^ Soysal, Ömer M (2015). "Association rule mining with mostly associated sequential patterns". Expert Systems with Applications. 42 (5): 2582–2592. doi:10.1016/j.eswa.2014.10.049.
- ^ Zhu, Yukun, et al. "Aligning books and movies: Towards story-like visual explanations by watching movies and reading books." Proceedings of the IEEE international conference on computer vision. 2015.
- ^ Bowman, Samuel R.; Angeli, Gabor; Potts, Christopher; Manning, Christopher D. (2015). "A large annotated corpus for learning natural language inference". arXiv:1508.05326 [cs.CL].
- ^ "DSL Corpus Collection". ttg.uni-saarland.de. Retrieved 22 September 2017.
- ^ "Urban Dictionary Words and Definitions".
- ^ H. Elsahar, P. Vougiouklis, A. Remaci, C. Gravier, J. Hare, F. Laforest, E. Simperl, "T-REx: A Large Scale Alignment of Natural Language with Knowledge Base Triples", Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018).
- ^ Wang, Alex; Singh, Amanpreet; Michael, Julian; Hill, Felix; Levy, Omer; Bowman, Samuel R. (2018). "GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding". arXiv:1804.07461 [cs.CL].
- ^ "Computers Are Learning to Read—But They're Still Not So Smart". Wired. Retrieved 29 December 2019.
- ^ "GLUE Benchmark". gluebenchmark.com. Retrieved 25 February 2019.
- ^ Quan, Hoang Lam; Quang, Duy Le; Van Kiet, Nguyen; Ngan, Luu-Thuy Nguyen. "UIT-ViIC: A Dataset for the First Evaluation on Vietnamese Image Captioning".
- ^ To, Quoc Huy; Nguyen, Van Kiet; Nguyen, Luu Thuy Ngan; Nguyen, Gia Tuan Anh (2020). "Gender Prediction Based on Vietnamese Names with Machine Learning Techniques". Proceedings of the 4th International Conference on Natural Language Processing and Information Retrieval. pp. 55–60. arXiv:2010.10852. doi:10.1145/3443279.3443309. ISBN 9781450377607. S2CID 224814110.
- ^ Nguyen, Luan Thanh; Van Nguyen, Kiet; Nguyen, Ngan Luu-Thuy (18 March 2021). "Constructive and Toxic Speech Detection for Open-Domain Social Media Comments in Vietnamese". Advances and Trends in Artificial Intelligence. Artificial Intelligence Practices. Lecture Notes in Computer Science. Vol. 12798. pp. 572–583. arXiv:2103.10069. doi:10.1007/978-3-030-79457-6_49. ISBN 978-3-030-79456-9. S2CID 232269671.
- ^ Saxton, David, et al. "Analysing Mathematical Reasoning Abilities of Neural Models." International Conference on Learning Representations. 2018.
- ^ M. Versteegh, R. Thiollière, T. Schatz, X.-N. Cao, X. Anguera, A. Jansen, and E. Dupoux (2015). "The Zero Resource Speech Challenge 2015," in INTERSPEECH-2015.
- ^ M. Versteegh, X. Anguera, A. Jansen, and E. Dupoux, (2016). "The Zero Resource Speech Challenge 2015: Proposed Approaches and Results," in SLTU-2016.
- ^ Sakar, Betul Erdogdu; et al. (2013). "Collection and analysis of a Parkinson speech dataset with multiple types of sound recordings". IEEE Journal of Biomedical and Health Informatics. 17 (4): 828–834. doi:10.1109/jbhi.2013.2245674. PMID 25055311. S2CID 15491516.
- ^ Zhao, Shunan, et al. "Automatic detection of expressed emotion in Parkinson's disease." Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on. IEEE, 2014.
- ^ Used in: Hammami, Nacereddine, and Mouldi Bedda. "Improved tree model for Arabic speech recognition." Computer Science and Information Technology (ICCSIT), 2010 3rd IEEE International Conference on. Vol. 5. IEEE, 2010.
- ^ Maaten, Laurens. "Learning discriminative fisher kernels." Proceedings of the 28th International Conference on Machine Learning (ICML-11). 2011.
- ^ Cole, Ronald, and Mark Fanty. "Spoken letter recognition." Proc. Third DARPA Speech and Natural Language Workshop. 1990.
- ^ Chapelle, Olivier; Sindhwani, Vikas; Keerthi, Sathiya S. (2008). "Optimization techniques for semi-supervised support vector machines" (PDF). The Journal of Machine Learning Research. 9: 203–233.
- ^ Kudo, Mineichi; Toyama, Jun; Shimbo, Masaru (1999). "Multidimensional curve classification using passing-through regions". Pattern Recognition Letters. 20 (11): 1103–1111. Bibcode:1999PaReL..20.1103K. CiteSeerX 10.1.1.46.2515. doi:10.1016/s0167-8655(99)00077-x.
- ^ Jaeger, Herbert; et al. (2007). "Optimization and applications of echo state networks with leaky-integrator neurons". Neural Networks. 20 (3): 335–352. doi:10.1016/j.neunet.2007.04.016. PMID 17517495.
- ^ Tsanas, Athanasios; et al. (2010). "Accurate telemonitoring of Parkinson's disease progression by noninvasive speech tests". IEEE Transactions on Biomedical Engineering (Submitted manuscript). 57 (4): 884–893. doi:10.1109/tbme.2009.2036000. PMID 19932995. S2CID 7382779.
- ^ Clifford, Gari D.; Clifton, David (2012). "Wireless technology in disease management and medicine". Annual Review of Medicine. 63: 479–492. doi:10.1146/annurev-med-051210-114650. PMID 22053737.
- ^ Zue, Victor; Seneff, Stephanie; Glass, James (1990). "Speech database development at MIT: TIMIT and beyond". Speech Communication. 9 (4): 351–356. doi:10.1016/0167-6393(90)90010-7.
- ^ Kapadia, Sadik, Valtcho Valtchev, and S. J. Young. "MMI training for continuous phoneme recognition on the TIMIT database." Acoustics, Speech, and Signal Processing, 1993. ICASSP-93., 1993 IEEE International Conference on. Vol. 2. IEEE, 1993.
- ^ Halabi, Nawar (2016). Modern Standard Arabic Phonetics for Speech Synthesis (PDF) (PhD Thesis). University of Southampton, School of Electronics and Computer Science.
- ^ Ardila, Rosana; Branson, Megan; Davis, Kelly; Henretty, Michael; Kohler, Michael; Meyer, Josh; Morais, Reuben; Saunders, Lindsay; Tyers, Francis M.; Weber, Gregor (13 December 2019). "Common Voice: A Massively-Multilingual Speech Corpus". arXiv:1912.06670v2 [cs.CL].
- ^ "The LJ Speech Dataset". keithito.com. Retrieved 13 April 2022.
- ^ Ghandoura, Abdulkader; Hjabo, Farouk; Al Dakkak, Oumayma (June 2021). "Building and benchmarking an Arabic Speech Commands dataset for small-footprint keyword spotting". Engineering Applications of Artificial Intelligence. 102: 104267. doi:10.1016/j.engappai.2021.104267. ISSN 0952-1976. S2CID 235637809.
- ^ Zhou, Fang, Q. Claire, and Ross D. King. "Predicting the geographical origin of music." Data Mining (ICDM), 2014 IEEE International Conference on. IEEE, 2014.
- ^ Saccenti, Edoardo; Camacho, José (2015). "On the use of the observation-wise k-fold operation in PCA cross-validation". Journal of Chemometrics. 29 (8): 467–478. doi:10.1002/cem.2726. hdl:10481/55302. S2CID 62248957.
- ^ Bertin-Mahieux, Thierry, et al. "The million song dataset." ISMIR 2011: Proceedings of the 12th International Society for Music Information Retrieval Conference, 24–28 October 2011, Miami, Florida. University of Miami, 2011.
- ^ Henaff, Mikael; et al. (2011). "Unsupervised learning of sparse features for scalable audio classification" (PDF). ISMIR. 11.
- ^ Rafii, Zafar (2017). "Music". MUSDB18 – a corpus for music separation. doi:10.5281/zenodo.1117372.
- ^ Defferrard, Michaël; Benzi, Kirell; Vandergheynst, Pierre; Bresson, Xavier (6 December 2016). "FMA: A Dataset For Music Analysis". arXiv:1612.01840 [cs.SD].
- ^ Esposito, Roberto; Radicioni, Daniele P. (2009). "Carpediem: Optimizing the viterbi algorithm and applications to supervised sequential learning" (PDF). The Journal of Machine Learning Research. 10: 1851–1880.
- ^ Sourati, Jamshid; et al. (2016). "Classification Active Learning Based on Mutual Information". Entropy. 18 (2): 51. Bibcode:2016Entrp..18...51S. doi:10.3390/e18020051.
- ^ Salamon, Justin; Jacoby, Christopher; Bello, Juan Pablo. "A dataset and taxonomy for urban sound research." Proceedings of the ACM International Conference on Multimedia. ACM, 2014.
- ^ Lagrange, Mathieu; Lafay, Grégoire; Rossignol, Mathias; Benetos, Emmanouil; Roebel, Axel (2015). "An evaluation framework for event detection using a morphological model of acoustic scenes". arXiv:1502.00141 [stat.ML].
- ^ Gemmeke, Jort F., et al. "Audio Set: An ontology and human-labeled dataset for audio events." IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). 2017.
- ^ "Watch out, birders: Artificial intelligence has learned to spot birds from their songs". Science | AAAS. 18 July 2018. Retrieved 22 July 2018.
- ^ "Bird Audio Detection challenge". Machine Listening Lab at Queen Mary University. 3 May 2016. Retrieved 22 July 2018.
- ^ Wichern, Gordon; Antognini, Joe; Flynn, Michael; Licheng Richard Zhu; McQuinn, Emmett; Crow, Dwight; Manilow, Ethan; Jonathan Le Roux (2019). "WHAM!: Extending Speech Separation to Noisy Environments". arXiv:1907.01160 [cs.SD].
- ^ Drossos, K., Lipping, S., and Virtanen, T. "Clotho: An Audio Captioning Dataset" IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). 2020.
- ^ Drossos, K., Lipping, S., and Virtanen, T. (2019). Clotho dataset (Version 1.0) [Data set]. Zenodo. http://doi.org/10.5281/zenodo.3490684
- ^ The CAIDA UCSD Dataset on the Witty Worm – 19–24 March 2004, http://www.caida.org/data/passive/witty_worm_dataset.xml
- ^ Chen, Zesheng, and Chuanyi Ji. "Optimal worm-scanning method using vulnerable-host distributions." International Journal of Security and Networks 2.1–2 (2007): 71–80.
- ^ Kachuee, Mohamad, et al. "Cuff-less high-accuracy calibration-free blood pressure estimation using pulse transit time[permanent dead link]." Circuits and Systems (ISCAS), 2015 IEEE International Symposium on. IEEE, 2015.
- ^ PhysioBank, PhysioToolkit. "PhysioNet: components of a new research resource for complex physiologic signals." Circulation. v101 i23. e215-e220.
- ^ Vergara, Alexander; et al. (2012). "Chemical gas sensor drift compensation using classifier ensembles". Sensors and Actuators B: Chemical. 166: 320–329. Bibcode:2012SeAcB.166..320V. doi:10.1016/j.snb.2012.01.074.
- ^ Korotcenkov, G.; Cho, B. K. (2014). "Engineering approaches to improvement of conductometric gas sensor parameters. Part 2: Decrease of dissipated (consumable) power and improvement stability and reliability". Sensors and Actuators B: Chemical. 198: 316–341. Bibcode:2014SeAcB.198..316K. doi:10.1016/j.snb.2014.03.069.
- ^ Quinlan, John R (1992). "Learning with continuous classes" (PDF). 5th Australian Joint Conference on Artificial Intelligence. 92.
- ^ Merz, Christopher J.; Pazzani, Michael J. (1999). "A principal components approach to combining regression estimates". Machine Learning. 36 (1–2): 9–32. doi:10.1023/a:1007507221352.
- ^ Torres-Sospedra, Joaquin, et al. "UJIIndoorLoc-Mag: A new database for magnetic field-based localization problems." Indoor Positioning and Indoor Navigation (IPIN), 2015 International Conference on. IEEE, 2015.
- ^ Berkvens, Rafael, Maarten Weyn, and Herbert Peremans. "Mean Mutual Information of Probabilistic Wi-Fi Localization." Indoor Positioning and Indoor Navigation (IPIN), 2015 International Conference on. Banff, Canada: IPIN. 2015.
- ^ Paschke, Fabian, et al. "Sensorlose Zustandsüberwachung an Synchronmotoren."Proceedings. 23. Workshop Computational Intelligence, Dortmund, 5.-6. Dezember 2013. KIT Scientific Publishing, 2013.
- ^ Lessmeier, Christian, et al. "Data Acquisition and Signal Analysis from Measured Motor Currents for Defect Detection in Electromechanical Drive Systems."
- ^ Ugulino, Wallace, et al. "Wearable computing: Accelerometers’ data classification of body postures and movements Archived 25 September 2020 at the Wayback Machine." Advances in Artificial Intelligence-SBIA 2012. Springer Berlin Heidelberg, 2012. 52–61.
- ^ Schneider, Jan; et al. (2015). "Augmenting the senses: a review on sensor-based learning support". Sensors. 15 (2): 4097–4133. Bibcode:2015Senso..15.4097S. doi:10.3390/s150204097. PMC 4367401. PMID 25679313.
- ^ Madeo, Renata CB, Clodoaldo AM Lima, and Sarajane M. Peres. "Gesture unit segmentation using support vector machines: segmenting gestures from rest positions." Proceedings of the 28th Annual ACM Symposium on Applied Computing. ACM, 2013.
- ^ Lun, Roanna; Zhao, Wenbing (2015). "A survey of applications and human motion recognition with Microsoft Kinect". International Journal of Pattern Recognition and Artificial Intelligence. 29 (5): 1555008. doi:10.1142/s0218001415550083.
- ^ Theodoridis, Theodoros, and Huosheng Hu. "Action classification of 3d human models using dynamic ANNs for mobile robot surveillance Archived 6 August 2019 at the Wayback Machine."Robotics and Biomimetics, 2007. ROBIO 2007. IEEE International Conference on. IEEE, 2007.
- ^ Etemad, Seyed Ali, and Ali Arya. "3D human action recognition and style transformation using resilient backpropagation neural networks." Intelligent Computing and Intelligent Systems, 2009. ICIS 2009. IEEE International Conference on. Vol. 4. IEEE, 2009.
- ^ Altun, Kerem; Barshan, Billur; Tunçel, Orkun (2010). "Comparative study on classifying human activities with miniature inertial and magnetic sensors". Pattern Recognition. 43 (10): 3605–3620. Bibcode:2010PatRe..43.3605A. doi:10.1016/j.patcog.2010.04.019. hdl:11693/11947.
- ^ Nathan, Ran; et al. (2012). "Using tri-axial acceleration data to identify behavioral modes of free-ranging animals: general concepts and tools illustrated for griffon vultures". The Journal of Experimental Biology. 215 (6): 986–996. doi:10.1242/jeb.058602. PMC 3284320. PMID 22357592.
- ^ Anguita, Davide, et al. "Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine." Ambient assisted living and home care. Springer Berlin Heidelberg, 2012. 216–223.
- ^ Su, Xing; Tong, Hanghang; Ji, Ping (2014). "Activity recognition with smartphone sensors". Tsinghua Science and Technology. 19 (3): 235–249. doi:10.1109/tst.2014.6838194. S2CID 62751498.
- ^ Kadous, Mohammed Waleed. Temporal classification: Extending the classification paradigm to multivariate time series. Diss. The University of New South Wales, 2002.
- ^ Graves, Alex, et al. "Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks." Proceedings of the 23rd international conference on Machine learning. ACM, 2006.
- ^ Velloso, Eduardo, et al. "Qualitative activity recognition of weight lifting exercises."Proceedings of the 4th Augmented Human International Conference. ACM, 2013.
- ^ Mortazavi, Bobak Jack, et al. "Determining the single best axis for exercise repetition recognition and counting on smartwatches Archived 4 November 2021 at the Wayback Machine." Wearable and Implantable Body Sensor Networks (BSN), 2014 11th International Conference on. IEEE, 2014.
- ^ Sapsanis, Christos, et al. "Improving EMG based Classification of basic hand movements using EMD." Engineering in Medicine and Biology Society (EMBC), 2013 35th Annual International Conference of the IEEE. IEEE, 2013.
- ^ a b Andrianesis, Konstantinos; Tzes, Anthony (2015). "Development and control of a multifunctional prosthetic hand with shape memory alloy actuators". Journal of Intelligent & Robotic Systems. 78 (2): 257–289. doi:10.1007/s10846-014-0061-6. S2CID 207174078.
- ^ Banos, Oresti; et al. (2014). "Dealing with the effects of sensor displacement in wearable activity recognition". Sensors. 14 (6): 9995–10023. Bibcode:2014Senso..14.9995B. doi:10.3390/s140609995. PMC 4118358. PMID 24915181.
- ^ Stisen, Allan, et al. "Smart Devices are Different: Assessing and MitigatingMobile Sensing Heterogeneities for Activity Recognition."Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems. ACM, 2015.
- ^ Bhattacharya, Sourav, and Nicholas D. Lane. "From Smart to Deep: Robust Activity Recognition on Smartwatches using Deep Learning."
- ^ Bacciu, Davide; et al. (2014). "An experimental characterization of reservoir computing in ambient assisted living applications". Neural Computing and Applications. 24 (6): 1451–1464. doi:10.1007/s00521-013-1364-4. hdl:11568/237959. S2CID 14124013.
- ^ Palumbo, Filippo; Barsocchi, Paolo; Gallicchio, Claudio; Chessa, Stefano; Micheli, Alessio (2013). "Multisensor Data Fusion for Activity Recognition Based on Reservoir Computing". Evaluating AAL Systems Through Competitive Benchmarking. Communications in Computer and Information Science. Vol. 386. pp. 24–35. doi:10.1007/978-3-642-41043-7_3. ISBN 978-3-642-41042-0.
- ^ Reiss, Attila, and Didier Stricker. "Introducing a new benchmarked dataset for activity monitoring."Wearable Computers (ISWC), 2012 16th International Symposium on. IEEE, 2012.
- ^ Roggen, Daniel, et al. "OPPORTUNITY: Towards opportunistic activity and context recognition systems." World of Wireless, Mobile and Multimedia Networks & Workshops, 2009. WoWMoM 2009. IEEE International Symposium on a. IEEE, 2009.
- ^ Kurz, Marc, et al. "Dynamic quantification of activity recognition capabilities in opportunistic systems." Vehicular Technology Conference (VTC Spring), 2011 IEEE 73rd. IEEE, 2011.
- ^ Sztyler, Timo, and Heiner Stuckenschmidt. "On-body localization of wearable devices: an investigation of position-aware activity recognition." Pervasive Computing and Communications (PerCom), 2016 IEEE International Conference on. IEEE, 2016.
- ^ Zhi, Ying Xuan; Lukasik, Michelle; Li, Michael H.; Dolatabadi, Elham; Wang, Rosalie H.; Taati, Babak (2018). "Automatic Detection of Compensation During Robotic Stroke Rehabilitation Therapy". IEEE Journal of Translational Engineering in Health and Medicine. 6: 2100107. doi:10.1109/JTEHM.2017.2780836. ISSN 2168-2372. PMC 5788403. PMID 29404226.
- ^ Dolatabadi, Elham; Zhi, Ying Xuan; Ye, Bing; Coahran, Marge; Lupinacci, Giorgia; Mihailidis, Alex; Wang, Rosalie; Taati, Babak (23 May 2017). "The toronto rehab stroke pose dataset to detect compensation during stroke rehabilitation therapy". Proceedings of the 11th EAI International Conference on Pervasive Computing Technologies for Healthcare. ACM. pp. 375–381. doi:10.1145/3154862.3154925. ISBN 9781450363631. S2CID 24581930.
- ^ "Toronto Rehab Stroke Pose Dataset".
- ^ Jung, Merel M.; Poel, Mannes; Poppe, Ronald; Heylen, Dirk K. J. (1 March 2017). "Automatic recognition of touch gestures in the corpus of social touch". Journal on Multimodal User Interfaces. 11 (1): 81–96. doi:10.1007/s12193-016-0232-9. ISSN 1783-8738. S2CID 1802116.
- ^ Jung, M.M. (Merel) (1 June 2016). "Corpus of Social Touch (CoST)". University of Twente. doi:10.4121/uuid:5ef62345-3b3e-479c-8e1d-c922748c9b29.
{{cite journal}}
: Cite journal requires|journal=
(help) - ^ Aeberhard, S., D. Coomans, and O. De Vel. "Comparison of classifiers in high dimensional settings." Dept. Math. Statist., James Cook Univ., North Queensland, Australia, Tech. Rep 92-02 (1992).
- ^ Basu, Sugato. "Semi-supervised clustering with limited background knowledge." AAAI. 2004.
- ^ Tüfekci, Pınar (2014). "Prediction of full load electrical power output of a base load operated combined cycle power plant using machine learning methods". International Journal of Electrical Power & Energy Systems. 60: 126–140. Bibcode:2014IJEPE..60..126T. doi:10.1016/j.ijepes.2014.02.027.
- ^ Kaya, Heysem, Pınar Tüfekci, and Fikret S. Gürgen. "Local and global learning methods for predicting power of a combined gas & steam turbine." International conference on emerging trends in computer and electronics engineering (ICETCEE'2012), Dubai. 2012.
- ^ Baldi, Pierre; Sadowski, Peter; Whiteson, Daniel (2014). "Searching for exotic particles in high-energy physics with deep learning". Nature Communications. 5: 2014. arXiv:1402.4735. Bibcode:2014NatCo...5.4308B. doi:10.1038/ncomms5308. PMID 24986233. S2CID 195953.
- ^ a b Baldi, Pierre; Sadowski, Peter; Whiteson, Daniel (2015). "Enhanced Higgs Boson to τ+ τ− Search with Deep Learning". Physical Review Letters. 114 (11): 111801. arXiv:1410.3469. Bibcode:2015PhRvL.114k1801B. doi:10.1103/physrevlett.114.111801. PMID 25839260. S2CID 2339142.
- ^ a b Adam-Bourdarios, C.; Cowan, G.; Germain-Renaud, C.; Guyon, I.; Kégl, B.; Rousseau, D. (2015). "The Higgs Machine Learning Challenge". Journal of Physics: Conference Series. 664 (7): 072015. Bibcode:2015JPhCS.664g2015A. doi:10.1088/1742-6596/664/7/072015.
- ^ Baldi, Pierre; Cranmer, Kyle; Faucett, Taylor; Sadowski, Peter; Whiteson, Daniel (2016). "Parameterized neural networks for high-energy physics". The European Physical Journal C. 76 (5): 235. arXiv:1601.07913. Bibcode:2016EPJC...76..235B. doi:10.1140/epjc/s10052-016-4099-4. S2CID 254108545.
- ^ Ortigosa, I.; Lopez, R.; Garcia, J. "A neural networks approach to residuary resistance of sailing yachts prediction". Proceedings of the International Conference on Marine Engineering MARINE. 2007.
- ^ Gerritsma, J., R. Onnink, and A. Versluis.Geometry, resistance and stability of the delft systematic yacht hull series. Delft University of Technology, 1981.
- ^ Liu, Huan, and Hiroshi Motoda. Feature extraction, construction and selection: A data mining perspective. Springer Science & Business Media, 1998.
- ^ Reich, Yoram. Converging to Ideal Design Knowledge by Learning. [Carnegie Mellon University], Engineering Design Research Center, 1989.
- ^ Todorovski, Ljupčo; Džeroski, Sašo (1999). "Experiments in Meta-level Learning with ILP". Principles of Data Mining and Knowledge Discovery. Lecture Notes in Computer Science. Vol. 1704. pp. 98–106. doi:10.1007/978-3-540-48247-5_11. ISBN 978-3-540-66490-1. S2CID 39382993.
- ^ Wang, Yong. A new approach to fitting linear models in high dimensional spaces. Diss. The University of Waikato, 2000.
- ^ Kibler, Dennis; Aha, David W.; Albert, Marc K. (1989). "Instance-based prediction of real-valued attributes". Computational Intelligence. 5 (2): 51–57. doi:10.1111/j.1467-8640.1989.tb00315.x. S2CID 40800413.
- ^ Palmer, Christopher R., and Christos Faloutsos. "Electricity based external similarity of categorical attributes." Advances in Knowledge Discovery and Data Mining. Springer Berlin Heidelberg, 2003. 486–500.
- ^ Tsanas, Athanasios; Xifara, Angeliki (2012). "Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools". Energy and Buildings. 49: 560–567. Bibcode:2012EneBu..49..560T. doi:10.1016/j.enbuild.2012.03.003.
- ^ De Wilde, Pieter (2014). "The gap between predicted and measured energy performance of buildings: A framework for investigation". Automation in Construction. 41: 40–49. doi:10.1016/j.autcon.2014.02.009.
- ^ Brooks, Thomas F., D. Stuart Pope, and Michael A. Marcolini. Airfoil self-noise and prediction. Vol. 1218. National Aeronautics and Space Administration, Office of Management, Scientific and Technical Information Division, 1989.
- ^ Draper, David. "Assessment and propagation of model uncertainty." Journal of the Royal Statistical Society, Series B (Methodological) (1995): 45–97.
- ^ Lavine, Michael (1991). "Problems in extrapolation illustrated with space shuttle O-ring data". Journal of the American Statistical Association. 86 (416): 919–921. doi:10.1080/01621459.1991.10475132.
- ^ Wang, Jun, Bei Yu, and Les Gasser. "Concept tree based clustering visualization with shaded similarity matrices." Data Mining, 2002. ICDM 2003. Proceedings. 2002 IEEE International Conference on. IEEE, 2002.
- ^ Pettengill, Gordon H.; Ford, Peter G.; Johnson, William T. K.; Raney, R. Keith; Soderblom, Laurence A. (1991). "Magellan: Radar Performance and Data Products". Science. 252 (5003): 260–265. Bibcode:1991Sci...252..260P. doi:10.1126/science.252.5003.260. PMID 17769272. S2CID 43398343.
- ^ a b Aharonian, F.; et al. (2008). "Energy spectrum of cosmic-ray electrons at TeV energies". Physical Review Letters. 101 (26): 261104. arXiv:0811.3894. Bibcode:2008PhRvL.101z1104A. doi:10.1103/PhysRevLett.101.261104. hdl:2440/51450. PMID 19437632. S2CID 41850528.
- ^ Bock, R. K.; et al. (2004). "Methods for multidimensional event classification: a case study using images from a Cherenkov gamma-ray telescope". Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment. 516 (2): 511–528. Bibcode:2004NIMPA.516..511B. doi:10.1016/j.nima.2003.08.157.
- ^ Li, Jinyan; et al. (2004). "Deeps: A new instance-based lazy discovery and classification system". Machine Learning. 54 (2): 99–124. doi:10.1023/b:mach.0000011804.08528.7d.
- ^ Villaescusa-Navarro, Francisco; al., et (2022). "The CAMELS Multifield Data Set: Learning the Universe's Fundamental Parameters with Artificial Intelligence". The Astrophysical Journal Supplement Series. 259 (2): 61. arXiv:2109.10915. Bibcode:2022ApJS..259...61V. doi:10.3847/1538-4365/ac5ab0. S2CID 237604997.
- ^ Siebert, Lee, and Tom Simkin. "Volcanoes of the world: an illustrated catalog of Holocene volcanoes and their eruptions." (2014).
- ^ Sikora, Marek; Wróbel, Łukasz (2010). "Application of rule induction algorithms for analysis of data collected by seismic hazard monitoring systems in coal mines". Archives of Mining Sciences. 55 (1): 91–114.
- ^ Sikora, Marek, and Beata Sikora. "Rough natural hazards monitoring." Rough Sets: Selected Methods and Applications in Management and Engineering. Springer London, 2012. 163–179.
- ^ Addor, Nans; Newman, Andrew J.; Mizukami, Naoki; Clark, Martyn P. (20 October 2017). "The CAMELS data set: catchment attributes and meteorology for large-sample studies". Hydrology and Earth System Sciences. 21 (10): 5293–5313. Bibcode:2017HESS...21.5293A. doi:10.5194/hess-21-5293-2017. ISSN 1607-7938.