formerly Computers in Cardiology
The PhysioNet/CinC Challenge
Background
While planning CinC 2000, local hosts Roger Mark and George Moody proposed to organize an activity that would make effective use of their newly-established PhysioNet web site to stimulate rapid progress on an unsolved problem of practical clinical significance. A timely contribution of data made it possible to create the first PhysioNet/CinC Challenge, which attracted the attention of more than a dozen teams to the subject of detecting sleep apnea from the ECG. Their efforts were broadly successful, they discussed their findings at CinC 2000, and an annual tradition was born.
PhysioNet offers free access via the web to large and growing collections of recorded physiologic signals and related open-source software. Originally established under the auspices of the NIH's National Center for Research Resources, PhysioNet has been funded since September 2007 under a cooperative agreement with the NIH's National Institute of Biomedical Imaging and Bioengineering (NIBIB), and with the NIH's National Institute of General Medical Sciences (NIGMS).
In complementary ways, PhysioNet and Computing in Cardiology catalyze and support scientific communication and collaboration between basic and clinical scientists. The annual meetings of CinC are gatherings of researchers from many nations and disciplines, bridging the geographic and specialty chasms that separate understanding from practice, while PhysioNet provides on-line data and software resources that support collaborations of basic and clinical researchers throughout the year. The annual PhysioNet/CinC Challenges seek to provide stimulating yet friendly competitions, while at the same time offering both specialists and non-specialists alike opportunities to make progress on significant open problems whose solutions may be of profound clinical value. The use of shared data provided via PhysioNet makes it possible for participants to work independently toward a common objective. At CinC, participants can make meaningful results-based comparisons of their methods; lively and well-informed discussions are the norm at scientific sessions dedicated to these challenges. Discovery of the complementary strengths of diverse approaches to a problem when coupled with deep understanding of that problem frequently sparks new collaborations and opportunities for further study.
A new challenge topic is announced each year on the Challenge page at PhysioNet. The PhysioNet team assembles and posts the raw materials needed to begin work. Usually, these raw materials consist of a collection of data to be analyzed; the analyses are provided for a subset of the data (the "learning set") in each case, and the challenge is to analyze the remaining data (the "test set").
What will be the topic of the next challenge? It might be image analysis, or simulation, or forecasting.... An ideal challenge problem is interesting, clinically important, and possible to study using available materials that have not been widely circulated previously. Moreover, there must be an objective way to evaluate the quality of a challenge entry (for an analysis problem, this usually means there must be a known set of correct analyses of the data, i.e., a "gold standard" against which entries can be compared). You are invited to submit your ideas for future challenge topics to the Challenge organizer, George Moody; suggestions accompanying relevant data are particularly welcome.
PhysioNet/CinC Challenge Awards
A generous contribution from the family of Solange Akselrod has permitted the Board of Computing in Cardiology to double the Challenge award fund in 2009 and 2010. Solange was an early and fervent supporter of these Challenges as a member of the CinC Board of Directors, and a mentor to participants in several of them.
Challenges are open to all. An important milestone for participants is the deadline for submitting abstracts for CinC, which is usually 1 May each year. Those wishing to qualify as official entrants, with eligibility for awards, must submit an abstract describing their work as well as an entry for scoring by this deadline. A limited number of revised entries may be submitted between 1 May and the final challenge deadline in early September. Eligibility for awards also requires participants to present their work in a scientific session of CinC.
Most Challenges are presented as two events (often a narrowly-defined question and a more general one). In each event, an award is offered for the best solution obtained by any eligible participant, and another award for the best open-source solution (i.e., the best solution for which the software written to obtain the solution is contributed by its author in open-source form to support further study). If an open-source solution is best overall, its author wins both awards for the event.
PhysioNet/CinC Challenge Award Winners
2009: Predicting Acute Hypotensive Episodes
- Event 1 (general and open-source):
- Xiaoxiao Chen, Michigan State University, Lansing, Michigan, USA
- Event 2 (general and open-source):
- Jorge Henriques, University of Coimbra, Coimbra, Portugal
2008: Detecting and Quantifying T-Wave Alternans
- General:
- Giovanni Bortolan, ISIB-CNR, Padova, Italy, and
Ivaylo Christov, Bulgarian Academy of Sciences, Sofia, Bulgaria - Open-source:
- Alexander Khaustov, Incart, St. Petersburg, Russia
2007: Electrocardiographic Imaging of Myocardial Infarction
- Event 1:
- Hamid SadAbadi, KN Toosi University, Teheran, Iran
- Event 2:
- Mohamed Mneimneh, Marquette University, Milwaukee, Wisconsin, USA
- Event 3 (tie):
- Masood Ghasemi, KN Toosi University, Teheran, Iran
- Mohamed Mneimneh, Marquette University, Milwaukee, Wisconsin, USA
- Hamid SadAbadi, KN Toosi University, Teheran, Iran
- Mohamed Mneimneh, Marquette University, Milwaukee, Wisconsin, USA
- Event 4 (tie):
- Mohamed Mneimneh, Marquette University, Milwaukee, Wisconsin, USA
- Hamid SadAbadi, KN Toosi University, Teheran, Iran
2006: QT Interval Measurement
- Division 1 (manual/semi-automated):
- Mariano Llamedo Soria, Universidad Tecnologica Nacional FRBA, Buenos Aires, Argentina
- Division 2 (automated, general):
- Dieter Hayn, ARC Seibersdorf Research GmbH, Graz, Austria
- Division 3 (automated, open-source):
- Yuri Chesnokov, Cambridge University, Cambridge, UK
2005: The First Five Challenges Revisited
- Richard Povinelli, Marquette University, Milwaukee, Wisconsin, USA
- C Raab, University of Potsdam, Potsdam, Germany
2004: Spontaneous Termination of Atrial Fibrillation
- Event A (general):
- Dieter Hayn, ARC Seibersdorf Research GmbH, Graz, Austria
- Event A (open-source):
- Maurizio Varanini, CNR-ICP, Pisa, Italy
- Event B (general and open-source):
- Federico Cantini, CNR-ICP, Pisa, Italy
2003: Distinguishing Ischemic from Non-Ischemic ST Changes
- Philip Langley, University of Newcastle upon Tyne, Newcastle, UK
2002: RR Interval Time Series Modeling
- Event 1:
- DC Lin, Ryerson University, Toronto, Canada
- Event 2:
- Albert C-C Yang, National Yang-Ming University, Taipei, Taiwan
2001: Predicting Paroxysmal Atrial Fibrillation
- Event 1:
- Günther Schreier, ARC Seibersdorf, Graz, Austria
- Event 2:
- Günther Schreier, ARC Seibersdorf, Graz, Austria
2000: Detecting Sleep Apnea from the ECG
- Event 1:
- Murray Jarvis, Caltech, Pasadena, California, USA
- Event 2:
- James McNames, Portland State University, Portland, Oregon, USA