Predicting tumor marker outcomes with Monte Carlo simulations

Jules J. Berman, Ph.D., M.D.Program Director for Pathology Informatics, CDP/NCI

Submitted, August 22, 2003

Presented, APIII, Pittsburgh, PA, Oct 8-10, 2003

All the information from this presentation as well as the source code is at:

http://65.222.228.150/jjb/randab.htm

Genome and proteome research have promised a revolution in tumor diagnosis. The revolution has not arrived.

In fact, only a handful of new markers have appeared in the past several years. A simple thought experiment demonstrates the problem.

Why do promising new prognostic/predictive markers fail? Consider this hypothetical case:

In a retrospective study, Dr. X demonstrated a "perfect" tumor marker that never failed to distinguish between two tumor variants (aggressive and indolent) with identical morphology.

In this example, an aggressive variant grows 10 times as fast and metastasizes at ten times the rate of the indolent variant with the same morphology.

In a prospective trial of the same marker, 200 tumors are excised at the time of clinical detection (tumor size 2 cm).

Dr. X finds that 100 of the tumors stain as "indolent variants" and 100 tumors stain as "aggressive variants".

The trials follows all 200 patients, determining survival at ten years. At the end of the trial, there is no survival difference between patients with "indolent variants" and patients with "aggressive variants".

The marker is considered a total failure, with millions of dollars wasted on the prospective trial.

HOW IS THIS BE POSSIBLE?

Lets look at a hypothetical scenario.

Hence, there was no difference in the survival outcome between the tumor variants.

This surprising result might occur in any prospective trial that starts at the time of tumor excision (e.g. radical prostatectomy).

Dr. X may have benefited from a simulation model designed to predict outcomes from a set of biological conditions and restraints.

What’s wrong with this analysis?

There’s no reason to think that it’s possible to have a tumor variant that grows at ten times the rate with 1/10th the chance of metastasis per unit time.

It’s possible to make hypothetical statements that have no realistic representation.

How do you avoid this?

You need to work from growth models.

Luckily, computers are just now powerful enough to model growth of tumors, starting from a single cell, and following the tumor growth until it reaches many millions of cells, calculating growth for each and every cell in the populations for each and every cell generation.

Perl scripts are available at the web site. You can feel free to play with the scripts and modify them to your heart’s content.

They’re pretty well annotated.

Basically, the Monte Carlo method used in the scripts can be explained in two lines of code.

$badchance = int(rand(10000)) +1;

if ($badchance == 10000)

{ $badoutcome, etc. etc. #pseudocode

“….. the act of putting your thoughts together into a coherent model often forces you to give up fondly held beliefs.The results is that people who have understood even the simplest, mosttrivial-sounding economic models are often far more sophisticated than people who know thousands of acts and hundreds of anecdotes, who can use plenty of big words, but have no coherent framework to organizetheir thoughts....”Paul Krugman. The Accidental Theorist, W.W. Norton & Company, NewYork, 1998.

Trialists depend on statisticians to determine whether the hypothesis is validated by the results of the trial.

However, statisticians can’t determine whether a trial has been sensibly designed to answer a valid question.

The trialist must determine this for him/herself, and the correct answer may defy “common sense.”

Computer simulations may be helpful when they model plausible outcomes unanticipated by human thought.

Perl is a free, open source, cross- platform language. All Perl scripts, along with explanatory text, are placed in the public domain and are available for download from: http://65.222.228.150/jjb/randab.htm