Magic Learner-JPL and others

FOR JPL

Study 1:

1. Predict LogicalDelivered, excluding EKLOC, inheritance.
2. Predict LogicalEKLOC, excluding delivered, and including inheritance.

Study 2:

1. Boring COCOMO using 1.1
2. Boring COCOMO using 1.2

Boring COCOMO ready and set to go.

Effort Predictions(val2):

so, we need a LOO experiment where we:

    0) cluster the dense columns 

    1)  take the "left out" example,  described in terms of:

                a) the new columns ("new" means "not traditional cocomo")

                b) its "class" e.g. "flight systems"
                c) act effort 

    2)  find the "left out" example's cluster and extract the LOC in that cluster
               - note: until step 5 we will ignore the actual effort

    3)  go to darren's rig and generate effort predictions (constrained by (2) and (1b))

    4) show the 50 to 70th percentile range of (3)

    5) mark on that  range the actual effort

    6) dance and sing if the the actual effort (from 1c) is in the range of (5

http://unbox.org/things/var/nave/lpj/out/master/effort_check_with_cocomo_50_70.xlsx

Graphs learnt on whole *old+new* with specific conditions(val0):

OTHER THAN JPL

Results for tables showing different techniques:

http://unbox.org/things/var/nave/lpj/out/master/*preptab*.dat

Contrast sets sorted:

http://unbox.org/things/var/nave/lpj/out/master/*contrast_sets_sorted.dat

Trees for different models:

http://unbox.org/things/var/nave/lpj/out/master/*dtree*.dat

Other graphs:

MRE with and without learning 20 records around cluster centroid:

JPL results

Results are here

ex: decs +=1 when risk in before > risk in after

Without any tricks:

before: 
w: both len: 500
     percentile          -risks         -effort        -defects         -months    #
             25             0.0          7539.9          1746.4            55.3    #
             50             0.3         12711.3          4332.0            65.2    #
             75             0.5         18808.1          8054.3            74.5    #
            100             2.4         39571.6         55458.4            98.6    #
after: 
wced: gened len: 500
     percentile          -risks         -effort        -defects         -months    #
             25             0.0          7171.9          1552.7            54.3    #
             50             0.3         12265.8          3394.4            64.1    #
             75             0.5         18948.0          6361.0            73.5    #
            100             2.4         33062.2         32483.0            94.0    #
###########################################################################
decs:  11 incs:  1 sames:  4

With rounded off to 2 decimal places:

flight
before: 
w: both len: 500
     percentile          -risks         -effort        -defects         -months    #
             25             0.0          7539.9          1746.4            55.3    #
             50             0.3         12711.3          4332.0            65.2    #
             75             0.5         18808.1          8054.3            74.5    #
            100             2.4         39571.6         55458.4            98.6    #
after: 
wced: gened len: 500
     percentile          -risks         -effort        -defects         -months    #
             25             0.0          8077.6          2545.1            56.2    #
             50             0.3         13000.4          4821.2            66.0    #
             75             0.5         19027.8          8807.1            74.1    #
            100             2.1         37703.9         61617.7            97.4    #
###########################################################################
decs: 4 incs: 9 sames: 3

With pruned leaves more than 0.3*root variance:

flight
before: 
w: both len: 500
     percentile          -risks         -effort        -defects         -months    #
             25             0.0          7539.9          1746.4            55.3    #
             50             0.3         12711.3          4332.0            65.2    #
             75             0.5         18808.1          8054.3            74.5    #
            100             2.4         39571.6         55458.4            98.6    #
after: 
wced: gened len: 500
     percentile          -risks         -effort        -defects         -months    #
             25             0.0          7167.8          1905.1            54.5    #
             50             0.0         13112.5          3694.2            65.2    #
             75             0.5         19143.8          9044.5            74.8    #
            100             2.1         42055.8         47935.8           101.8    #
###########################################################################
decs: 6 incs: 7 sames: 3

With near=50:

flight
before: 
w: both len: 500
     percentile          -risks         -effort        -defects         -months    #
             25             0.0          7539.9          1746.4            55.3    #
             50             0.3         12711.3          4332.0            65.2    #
             75             0.5         18808.1          8054.3            74.5    #
            100             2.4         39571.6         55458.4            98.6    #
after: 
wced: gened len: 500
     percentile          -risks         -effort        -defects         -months    #
             25             0.0          6628.1          2226.9            53.7    #
             50             0.3         12449.1          4711.6            63.9    #
             75             0.5         18024.6          8876.6            72.7    #
            100             2.4         36447.5         43329.9           100.9    #
###########################################################################
decs: 8 incs: 4 sames: 4

With near=20:

flight
before: 
w: both len: 500
     percentile          -risks         -effort        -defects         -months    #
             25             0.0          7539.9          1746.4            55.3    #
             50             0.3         12711.3          4332.0            65.2    #
             75             0.5         18808.1          8054.3            74.5    #
            100             2.4         39571.6         55458.4            98.6    #
after: 
wced: gened len: 500
     percentile          -risks         -effort        -defects         -months    #
             25             0.0          6445.3          1803.4            52.7    #
             50             0.3         13185.8          4169.2            65.8    #
             75             0.5         18991.1          9522.6            75.0    #
            100             2.4         35222.3         43783.5           100.7    #
###########################################################################
decs: 5 incs: 7 sames: 4

With near=10:

Notice the decrease in defects to half and also no increases by only decreases. good sign.

flight
before: 
w: both len: 500
     percentile          -risks         -effort        -defects         -months    #
             25             0.0          7539.9          1746.4            55.3    #
             50             0.3         12711.3          4332.0            65.2    #
             75             0.5         18808.1          8054.3            74.5    #
            100             2.4         39571.6         55458.4            98.6    #
after: 
wced: gened len: 500
     percentile          -risks         -effort        -defects         -months    #
             25             0.0          6576.3          1141.0            53.1    #
             50             0.0         12201.7          2659.5            63.1    #
             75             0.5         17473.0          4836.8            71.1    #
            100             2.1         37588.2         24027.8            95.9    #
###########################################################################
decs: 14 incs: 0 sames: 2

With near=5:

flight
before: 
w: both len: 500
     percentile          -risks         -effort        -defects         -months    #
             25             0.0          7539.9          1746.4            55.3    #
             50             0.3         12711.3          4332.0            65.2    #
             75             0.5         18808.1          8054.3            74.5    #
            100             2.4         39571.6         55458.4            98.6    #
after: 
wced: gened len: 500
     percentile          -risks         -effort        -defects         -months    #
             25             0.0          7270.6          1776.4            54.9    #
             50             0.0         13055.3          4027.3            65.3    #
             75             0.5         18314.6          7566.9            74.1    #
            100             2.4         42903.9         48402.4           103.2    #
###########################################################################
decs: 8 incs: 5 sames: 3

Summing up:

?Without --Without seems to decs are 11 and incs are 1, different than what I saw in the mail. Perhaps my edits after generating flight.csv might have caused that.

-Roundoff ---for generating ptiles: few clusters are not having any rows left after pruning rows off that dont obey the conditions. Every row violates atleast one condition in few clusters.

-Prune --Pruned leaves with variances over 0.3 that of root. ~3-4 leaves get eliminated.

+near when n = 10 --Works for a range of n when n is neither too high for excessive learning nor too low for insufficient learning.