- The queue is backing up - is the server down?
-
From time to time during the contest we will pause the queue to
perform some maintenance on the database used to store contest
results. At these times, the contest queue will be put on hold. We
don't expect these downtimes to be very long, and will try to announce
them in the Message of The Day (MOTD) at the main contest page.
If there are any problems with the queue which affect the performance
of any entries, after correcting the problem we will automatically
rerun the affected contest entries to produce valid scores.
- Do comments hurt performance?
-
Comments do not hurt performance. Each entry is Pcoded (see 'helpwin
pcode') before it is run. This removes the effects comments would
have on execution time.
- How is the contest run?
-
The contest is run using the
MATLAB Web Server as an interface between
your web browser and a MATLAB session which runs and scores the
entries. For more details, see our description of the Contest architecture.
The contest machine is a single-processor Windows NT computer with 128
MB RAM; contest entries are run one at a time.
- Is there a chat-room for contest
discussion?
-
There's a newsgroup called
comp.soft-sys.matlab; this Internet
newsgroup is an excellent place for MATLAB related discussion. We've
started a thread called "MATLAB Surveyor Contest"; feel free to post
comments about the contest for other users there. You can find the
contest by clicking on this link:
news:comp.soft-sys.matlab
If your newsreader isn't configured to read newsgroups, you can use a
web site such as Deja.com to read the newsgroup through your web
browser. You can reach Deja.com by clicking on this link:
http://www.deja.com
Please note that Deja.com isn't related to the MathWorks, so we can't
guarantee their service.
-
Will there be an analysis of the contest once it ends?
-
At the end of the contest we'll put together an analysis of how the
entries evolved over time. This includes various methods of analyzing
score vs. contest time and an analysis of the "lineage" of various
entries. We'll also put together a description of the workings of the
winning entry, and what gave it the edge over other entries. We'll
also make available the test suite that we used to run the contest.
-
How often is the contest held?
-
This is the second MATLAB contest that we've run on the Web. The first
was held in December 1998. We hope to continue to run the MATLAB contest
twice a year. Watch for announcements in
comp.soft-sys.matlab, in the MATLAB
Digest newsletter, or on www.mathworks.com,
- Can I see the test suite? Why doesn't
my entry pass your test suite? It does perfectly well on the cases
I've tried.
-
The test suite is not the same as the examples we've made available
for you to try out. It's possible that your code is having trouble
with some of our test cases.
We will release the test suite at the end of the contest, at which
time you can check the entries that failed to see what happened.
However, we don't plan to release any other information about the test
cases during the contest.
- Why do identical
submissions have different scores?
-
If there were no timing variability, we'd expect identical entries to
have the same CPU time and score. However, because of the variability
present in timing entries, there is some scoring variability between
runs of the same code, even with the same test case. This slight
difference in the CPU time of identical entries is caused by the
"noise" present when measuring CPU time. Since in this contest we're
only running 8 tests, it's possible that this timing variability will
affect your score slightly.
Also, entries relying on random numbers can perform differently,
depending on the sequence generated.
- How was the scoring method
determined?
-
The scoring system was chosen to weight the performance parameters in
a way that we felt was appropriate. This is somewhat arbitrary; as
what's appropriate isn't always objective; it's likely that the
rankings you see would be different if a different scoring function
were used.
- It's frustrating to have someone
modify your code just a bit and receive a better score. It seems as
if the best entries won't be submitted until the last minute, since no
one will want to have his/her entry beaten by a slight modification.
Why is the contest run this way?
-
One of the goals behind the MATLAB Contest is to give you an opportunity
learn more about MATLAB programming.
One of the best ways to learn is to view and try modifying other people's code.
It's not often that you have some of the world's best MATLAB programmer's
actively working to optimize your code!
From previous contests we've seen a recurring pattern - a particular
algorithm is submitted and then it successively improved and tweaked
by another contestant to take the top spot. This continues
until there is a new submission that makes a significant algorithmic
breakthrough, and the cycle continues.
This creative energy is clear throughout the contest, and not just at
the end. The top entries typically have been repeatedly modified by
several contestants. It's fair to say that the winning entry could
not have been written by any one person. In our last contest,
the lineage of the winning entry included direct contributions by at least
nine different people!
At the end of the contest we'll post an analysis of the entries
showing how the entries developed over the course of the contest.
This analysis includes highlights of the tweaking that went on in
successful entries. At this point you can more clearly see if
allowing code tweaking has hindered or helped the creativity involved
in the contest. From what we've seen, the tweaking has made the
contest more of a learning experience.
Although your contributions may not qualify you for one of our prizes,
we will be certain to acknowledge every contributor to the winning entry.
