Shown here are results for GALE/NSGA-II/SPEA2 for a single run each on the CDA simulation for the WMC3 framework (i.e. WMC3CDA).
Note that each single run evaluates every member of every population for comparison purposes, but with GALE, not every evaluation is necessary for optimization. Thus, each algorithm takes ~ about 4 hours to do a single run.
The results below are for a single run of the algorithm. A more complete analysis would require ~ 10 runs at the very least, or 20 to be minimally satisfying to put up a solid statistical comparison.
Results:
Dot Plots: 1
http://i.imgur.com/jyfLyDH.png
Line Plots:
GALE is on top. Bottom plots have messed up plot marker-choices, so read carefully.
http://i.imgur.com/LxkL6eh.png
Comments:
Although GALE reaches some lows not seen by the other algorithms, GALE has much larger variance, which could theoretically be fixed by removing random jiggles.
Note that the adjustments in making GALE deterministic (removing all randoms) are not used in these results.
Whats Next:
Need more Decision Variables!
- Different landing routes
- Radar Tower parameters?
Prune some objectives away? Focus on time lost instead of number of delayed/interrupted tasks.
NASA furloughs - no access to their servers for now. Need to get stuff working on my machines.
Showing posts with label cda. Show all posts
Showing posts with label cda. Show all posts
Tuesday, October 1, 2013
Tuesday, September 24, 2013
Bounce at "1 minus (east-west)"?
WMC3 - CDA
Recap Problem Statement: We're trying to use GALE to make safety-critical decisions in short time. We're using the CDA model from WMC3, built by Georgia Tech scholars, to learn what organizational structures within the cockpit (in terms of how the pilots should handle their taskload) best optimize objectives such as minimizing time of delay/interruptions in getting these tasks completed. This is a cognitive science domain.
Buggy dataset: http://i.imgur.com/GCZFxie.png. Takes about 12-15 minutes to run 100 times.
Comparing to results (example: http://i.imgur.com/znv2xzv.png) from http://cci.drexel.edu/NextGenAA/pdf/jcedm-2013-Pritchett-measuring%20fa.pdf. This is our main reference paper for sanity checks on the model.
Few obvious correlations. Sanity check.
Task: Figure out the bugs. Then we can take off.
GALE
What I did:
Took away all non-determinism (random injections).
Only randomness that exists is with regeneration of individuals - but this isn't really non-deterministic (probability bin distributions).
Results: No change.
Self-reflection: we're mutating some distance, usually going too far. Then we try to come back towards it, and we go too far. Repeat, bouncing back and forth across where we want to go.
Problem: How far do we really want to bounce to get to exactly where we want to go?
Solution: Our projection axis is typically 1.0 units long. In the optimal ideal world, east = 1.0, and west = 0.0 (or vice versa). Thus absolute difference is 1.0 - 0.0 = 1.0. Taking 1 minus this, this tells us how far we need to bounce. "1 - (east - west)".
Observations: In early generations, this bounce distance is on the order of 0.30. In later generations, it's very close to 0.0. In addition, once we get close to 0.0, we STAY there.
Hypotheses: In problems that are hard to optimize, we don't get close to 0.0. Bounce distance tells us on its own how good we're doing.
New Results:
Some standard Models (Osyczka2, Vienett2, Schaffer): http://i.imgur.com/MUUC2pD.png
POM3 Models: http://i.imgur.com/aSK1gWj.png
GALE POM3B Decision Chart: http://i.imgur.com/zoL8ZiW.png
Observations: Less variance in the dots. Clearer image than before (active v6 link: http://unbox.org/things/var/joe/active/active-v6.pdf)
Recap Problem Statement: We're trying to use GALE to make safety-critical decisions in short time. We're using the CDA model from WMC3, built by Georgia Tech scholars, to learn what organizational structures within the cockpit (in terms of how the pilots should handle their taskload) best optimize objectives such as minimizing time of delay/interruptions in getting these tasks completed. This is a cognitive science domain.
Buggy dataset: http://i.imgur.com/GCZFxie.png. Takes about 12-15 minutes to run 100 times.
Comparing to results (example: http://i.imgur.com/znv2xzv.png) from http://cci.drexel.edu/NextGenAA/pdf/jcedm-2013-Pritchett-measuring%20fa.pdf. This is our main reference paper for sanity checks on the model.
Few obvious correlations. Sanity check.
Task: Figure out the bugs. Then we can take off.
GALE
What I did:
Took away all non-determinism (random injections).
Only randomness that exists is with regeneration of individuals - but this isn't really non-deterministic (probability bin distributions).
Results: No change.
Self-reflection: we're mutating some distance, usually going too far. Then we try to come back towards it, and we go too far. Repeat, bouncing back and forth across where we want to go.
Problem: How far do we really want to bounce to get to exactly where we want to go?
Solution: Our projection axis is typically 1.0 units long. In the optimal ideal world, east = 1.0, and west = 0.0 (or vice versa). Thus absolute difference is 1.0 - 0.0 = 1.0. Taking 1 minus this, this tells us how far we need to bounce. "1 - (east - west)".
Observations: In early generations, this bounce distance is on the order of 0.30. In later generations, it's very close to 0.0. In addition, once we get close to 0.0, we STAY there.
Hypotheses: In problems that are hard to optimize, we don't get close to 0.0. Bounce distance tells us on its own how good we're doing.
New Results:
Some standard Models (Osyczka2, Vienett2, Schaffer): http://i.imgur.com/MUUC2pD.png
POM3 Models: http://i.imgur.com/aSK1gWj.png
GALE POM3B Decision Chart: http://i.imgur.com/zoL8ZiW.png
Observations: Less variance in the dots. Clearer image than before (active v6 link: http://unbox.org/things/var/joe/active/active-v6.pdf)
Subscribe to:
Posts (Atom)