Wednesday, May 28, 2014

Cluster-based Optimization

Update: Hedges Comparison

ant-1.4
[32, 31, 32, 30]
[54, 48, 44, 32]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
50 | 51 | 0.03 | 0

ant-1.5
[46, 44, 45, 43]
[90, 96, 59, 48]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
51 | 52 | 0.29 | 0

ant-1.6
[38, 36, 37, 36, 38, 36, 37, 35]
[48, 39, 41, 36, 50, 50, 57, 30]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
49 | 50 | 0.22 | 0

ant-1.7
[43, 45, 44, 44, 43, 44, 45, 43]
[114, 210, 76, 68, 59, 88, 77, 53]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
50 | 50 | 0.04 | 0

camel-1.2
[42, 43, 41, 44, 43, 42, 43, 41]
[97, 75, 65, 89, 59, 63, 117, 43]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
51 | 50 | 0.37 | 0

camel-1.4
[77, 76, 75, 77, 77, 75, 76, 75]
[106, 119, 127, 119, 109, 100, 137, 55]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
55 | 51 | 0.91 | 1

camel-1.6
[112, 106, 105, 114, 110, 108, 109, 108]
[137, 114, 138, 134, 129, 107, 153, 53]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
49 | 50 | 0.51 | 1

ivy-1.4
[29, 27, 28, 27]
[82, 58, 67, 34]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
42 | 52 | 1.68 | 2

ivy-2.0
[61, 60, 61, 59]
[116, 76, 114, 46]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
49 | 50 | 0.10 | 0

jedit-4.0
[35, 34, 35, 33, 35, 33, 34, 33]
[42, 45, 35, 37, 38, 40, 52, 17]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
57 | 55 | 0.22 | 0

jedit-4.1
[40, 38, 39, 38, 39, 37, 38, 37]
[41, 36, 34, 43, 38, 48, 56, 16]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
55 | 52 | 0.39 | 1

jedit-4.2
[40, 39, 40, 38, 40, 38, 39, 38]
[46, 46, 36, 48, 48, 59, 63, 21]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
53 | 52 | 0.10 | 0

jedit-4.3
[47, 46, 45, 46, 47, 45, 46, 45]
[106, 56, 48, 59, 51, 56, 102, 14]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
50 | 49 | 0.04 | 0

log4j-1.1
[35, 33, 34, 33]
[27, 21, 35, 26]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
62 | 57 | 0.35 | 0

log4j-1.2
[28, 27, 28, 26]
[65, 45, 64, 31]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
64 | 60 | 0.20 | 0

lucene-2.2
[50, 48, 49, 48]
[71, 66, 79, 31]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
51 | 54 | 0.80 | 1

lucene-2.4
[32, 31, 62, 62, 60]
[50, 50, 86, 102, 52]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
53 | 51 | 0.44 | 1

synapse-1.1
[40, 39, 40, 38]
[70, 46, 61, 45]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
54 | 52 | 0.25 | 0

synapse-1.2
[58, 54, 56, 54]
[76, 49, 77, 54]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
55 | 52 | 0.79 | 1

velocity-1.5
[50, 49, 49, 48]
[74, 45, 80, 15]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
53 | 52 | 0.39 | 0

velocity-1.6
[55, 53, 54, 52]
[76, 48, 89, 16]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
52 | 49 | 1.15 | 2

xalan-2.5
[93, 89, 91, 89, 91, 90, 91, 89]
[124, 103, 89, 86, 92, 105, 129, 75]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
50 | 50 | 0.03 | 0

xalan-2.6
[102, 100, 101, 99, 101, 100, 101, 99]
[115, 94, 87, 128, 119, 106, 149, 87]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
47 | 50 | 0.36 | 0

xalan-2.7
[113, 111, 112, 111, 111, 109, 110, 108]
[130, 117, 106, 102, 132, 129, 112, 81]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
73 | 63 | 0.64 | 1

xerces-1.3
[58, 54, 60, 50, 56, 54, 55, 53]
[75, 60, 73, 31, 54, 64, 62, 34]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
49 | 50 | 0.16 | 0

xerces-1.4
[59, 55, 61, 52, 58, 56, 57, 55]
[104, 77, 96, 48, 70, 84, 84, 25]
Cluster pD,pF mean AUC | Straw Cluster mean AUC | Hedges | Level
65 | 57 | 0.58 | 1


Explanation:

Results:

This doesn't always help us, (sometimes it hurts) but for some clusters, it REALLY works.

Key: (too verbose for each graph)
Optimized Results for each cluster        == colored Solid line with Stars
Default Param Results for each cluster  == colored Dashed line with Circles
No clustering, just optimize N-1, test N == Double-dashed Black line with Diamonds

























The AUCs:

ant-1.4 pD, pF AUCs (optimized/default)
No Clustering: 52
Cluster 0 50/49
Cluster 1 45/46
Cluster 2 49/51
Cluster 3 57/56
===============================================
ant-1.5 pD, pF AUCs (optimized/default)
No Clustering: 58
Cluster 0 40/50
Cluster 1 50/51
Cluster 2 55/53
Cluster 3 57/55
===============================================
ant-1.6 pD, pF AUCs (optimized/default)
No Clustering: 65
Cluster 0 50/50
Cluster 1 49/49
Cluster 2 49/49
Cluster 3 57/46
Cluster 4 46/47
Cluster 5 50/52
Cluster 6 49/47
Cluster 7 57/58
===============================================
ant-1.7 pD, pF AUCs (optimized/default)
No Clustering: 66
Cluster 0 53/52
Cluster 1 50/49
Cluster 2 44/48
Cluster 3 53/50
Cluster 4 54/51
Cluster 5 49/48
Cluster 6 46/50
Cluster 7 56/53
===============================================
camel-1.2 pD, pF AUCs (optimized/default)
No Clustering: 53
Cluster 0 50/50
Cluster 1 49/49
Cluster 2 50/50
Cluster 3 50/50
Cluster 4 50/50
Cluster 5 53/50
Cluster 6 55/54
Cluster 7 54/53
===============================================
camel-1.4 pD, pF AUCs (optimized/default)
No Clustering: 60
Cluster 0 60/52
Cluster 1 53/49
Cluster 2 50/48
Cluster 3 63/54
Cluster 4 53/56
Cluster 5 52/49
Cluster 6 55/52
Cluster 7 50/48
===============================================
camel-1.6 pD, pF AUCs (optimized/default)
No Clustering: 55
Cluster 0 51/49
Cluster 1 46/48
Cluster 2 47/49
Cluster 3 51/54
Cluster 4 47/50
Cluster 5 50/50
Cluster 6 54/52
Cluster 7 47/50
===============================================
ivy-1.4 pD, pF AUCs (optimized/default)
No Clustering: 58
Cluster 0 46/51
Cluster 1 33/61
Cluster 2 42/53
Cluster 3 39/43
===============================================
ivy-2.0 pD, pF AUCs (optimized/default)
No Clustering: 58
Cluster 0 47/49
Cluster 1 41/49
Cluster 2 49/48
Cluster 3 56/54
===============================================
jedit-4.0 pD, pF AUCs (optimized/default)
No Clustering: 65
Cluster 0 47/48
Cluster 1 50/49
Cluster 2 49/50
Cluster 3 52/47
Cluster 4 67/65
Cluster 5 69/58
Cluster 6 56/59
Cluster 7 69/62
===============================================
jedit-4.1 pD, pF AUCs (optimized/default)
No Clustering: 68
Cluster 0 48/48
Cluster 1 52/46
Cluster 2 52/47
Cluster 3 45/46
Cluster 4 42/49
Cluster 5 66/57
Cluster 6 57/60
Cluster 7 68/61
===============================================
jedit-4.2 pD, pF AUCs (optimized/default)
No Clustering: 68
Cluster 0 60/48
Cluster 1 47/46
Cluster 2 47/48
Cluster 3 50/59
Cluster 4 54/55
Cluster 5 61/58
Cluster 6 52/52
Cluster 7 56/52
===============================================
jedit-4.3 pD, pF AUCs (optimized/default)
No Clustering: 61
Cluster 0 49/49
Cluster 1 48/49
Cluster 2 47/48
Cluster 3 45/48
Cluster 4 40/46
Cluster 5 51/45
Cluster 6 58/54
Cluster 7 60/57
===============================================
log4j-1.1 pD, pF AUCs (optimized/default)
No Clustering: 75
Cluster 0 39/48
Cluster 1 52/47
Cluster 2 73/65
Cluster 3 73/69
===============================================
log4j-1.2 pD, pF AUCs (optimized/default)
No Clustering: 57
Cluster 0 47/54
Cluster 1 54/51
Cluster 2 58/56
Cluster 3 94/80
===============================================
lucene-2.2 pD, pF AUCs (optimized/default)
No Clustering: 57
Cluster 0 49/53
Cluster 1 54/51
Cluster 2 56/54
Cluster 3 55/56
===============================================
lucene-2.4 pD, pF AUCs (optimized/default)
No Clustering: 58
Cluster 0 51/52
Cluster 1 43/46
Cluster 2 54/52
Cluster 3 58/53
Cluster 4 52/50
===============================================
synapse-1.1 pD, pF AUCs (optimized/default)
No Clustering: 55
Cluster 0 49/51
Cluster 1 44/53
Cluster 2 48/51
Cluster 3 64/54
===============================================
synapse-1.2 pD, pF AUCs (optimized/default)
No Clustering: 62
Cluster 0 58/52
Cluster 1 51/50
Cluster 2 49/51
Cluster 3 54/55
===============================================
velocity-1.5 pD, pF AUCs (optimized/default)
No Clustering: 49
Cluster 0 50/50
Cluster 1 57/53
Cluster 2 57/55
Cluster 3 59/48
===============================================
velocity-1.6 pD, pF AUCs (optimized/default)
No Clustering: 58
Cluster 0 44/48
Cluster 1 54/50
Cluster 2 49/50
Cluster 3 53/50
===============================================
xalan-2.5 pD, pF AUCs (optimized/default)
No Clustering: 55
Cluster 0 50/50
Cluster 1 50/50
Cluster 2 51/49
Cluster 3 52/49
Cluster 4 48/50
Cluster 5 52/53
Cluster 6 47/47
Cluster 7 59/54
===============================================
xalan-2.6 pD, pF AUCs (optimized/default)
No Clustering: 56
Cluster 0 51/51
Cluster 1 53/49
Cluster 2 50/48
Cluster 3 56/54
Cluster 4 47/49
Cluster 5 47/49
Cluster 6 47/51
Cluster 7 40/47
===============================================
xalan-2.7 pD, pF AUCs (optimized/default)
No Clustering: 72
Cluster 0 53/51
Cluster 1 63/54
Cluster 2 58/40
Cluster 3 79/59
Cluster 4 81/64
Cluster 5 69/70
Cluster 6 88/79
Cluster 7 98/89
===============================================
xerces-1.3 pD, pF AUCs (optimized/default)
No Clustering: 48
Cluster 0 39/46
Cluster 1 61/49
Cluster 2 46/52
Cluster 3 33/50
Cluster 4 49/55
Cluster 5 46/48
Cluster 6 54/53
Cluster 7 52/51
===============================================
xerces-1.4 pD, pF AUCs (optimized/default)
No Clustering: 63
Cluster 0 50/50
Cluster 1 54/51
Cluster 2 60/54
Cluster 3 56/52
Cluster 4 67/54
Cluster 5 73/58
Cluster 6 68/58
Cluster 7 97/83
===============================================

Tuesday, May 27, 2014

Results technique tables j*

Just with last gen values:

Techniques         -effort         -months        -defects          -risks    #
           T0 m              29              67              14               2    #
  T3 C25 N100 m               2              10               3               0    #
     T9:jlast m               0              35               0              22    #
-------------------------------------------------------------------------------------
           T0 q              26              16              20               6    #
  T3 C25 N100 q               2               0               5               1    #
     T9:jlast q               3              23               4              38    #
-------------------------------------------------------------------------------------
           T0 w             100             100             100              20    #
  T3 C25 N100 w              31              21              47               4    #
     T9:jlast w              32             100              56             100    #
-------------------------------------------------------------------------------------
            100         2687.38            41.4        17612.38             8.6    #
              0          117.21            1.92          381.83            0.03    #

Different Techniques:

Techniques         -effort         -months        -defects          -risks    #
           T0 m              22              44              10               2    #
  T3 C25 N100 m               4               6               3               0    #
      T9:j/j_ m               2              23               1              16    #
   T9:j/j_l20 m               6              30               4               4    #
    T9:j/j_l6 m               3              33               2               9    #
 T9:j/j_m0.01 m               3              26               2               5    #
  T9:j/j_m0.5 m               7              26               5               8    #
  T9:j/j_m0.7 m               3              25               1               2    #
  T9:j/j_p300 m               5              45               2               7    #
  T9:j/j_p500 m               0              16               0               4    #
-------------------------------------------------------------------------------------
           T0 q              20              11              15               4    #
  T3 C25 N100 q               4               0               4               1    #
      T9:j/j_ q               5              15               3              28    #
   T9:j/j_l20 q              20              20               7              14    #
    T9:j/j_l6 q               6              22               6              16    #
 T9:j/j_m0.01 q               7               9               3              14    #
  T9:j/j_m0.5 q              11              13               6              11    #
  T9:j/j_m0.7 q               7              21               2               7    #
  T9:j/j_p300 q              10              30               4              30    #
  T9:j/j_p500 q               0               8               1               9    #
-------------------------------------------------------------------------------------
           T0 w              71              65              68              15    #
  T3 C25 N100 w              22              13              32               3    #
      T9:j/j_ w              24              65              38              74    #
   T9:j/j_l20 w              52              59              26              54    #
    T9:j/j_l6 w              31              74              29              66    #
 T9:j/j_m0.01 w              26              58              14              79    #
  T9:j/j_m0.5 w              67              66             100              55    #
  T9:j/j_m0.7 w             100              69              34              37    #
  T9:j/j_p300 w              43             100              23             100    #
  T9:j/j_p500 w               6              33               6              60    #
-------------------------------------------------------------------------------------
            100          3798.0           62.54         25832.5           11.55    #
              0           28.96            1.89          163.65            0.04    #

m = mutation rate
p = population size
l = lives of bstop/generations

Monday, May 19, 2014

Different Techniques Contrast Set Learner

Flight


Techniques         -effort         -months        -defects          -risks    #
           T0 m              14              59               5              10    #
 T3 C100 N100 m              11              44               9               5    #
  T3 C100 N50 m              12              53              10               2    #
  T3 C100 N75 m              10              47               6              10    #
  T3 C25 N100 m               0               9               0               3    #
   T3 C25 N50 m               4              23               3               2    #
   T3 C25 N75 m               0               9               0               2    #
  T3 C50 N100 m               2              19               3               2    #
   T3 C50 N50 m               5              38               4               0    #
   T3 C50 N75 m               6              30               4               6    #
-------------------------------------------------------------------------------------
           T0 q              12              15               8              26    #
 T3 C100 N100 q               9              10              11              17    #
  T3 C100 N50 q              12              15              14              17    #
  T3 C100 N75 q              10              13               9              23    #
  T3 C25 N100 q               0               0               1               5    #
   T3 C25 N50 q               4               6               5              10    #
   T3 C25 N75 q               0               0               1               6    #
  T3 C50 N100 q               2               3               4               9    #
   T3 C50 N50 q               5               8               5              11    #
   T3 C50 N75 q               6               7               6              14    #
-------------------------------------------------------------------------------------
           T0 w              50              88              45              89    #
 T3 C100 N100 w              62              75              99              76    #
  T3 C100 N50 w             100             100              97              55    #
  T3 C100 N75 w              61              80             100             100    #
  T3 C25 N100 w              18              19              21              20    #
   T3 C25 N50 w              30              42              51              35    #
   T3 C25 N75 w              16              19              28              23    #
  T3 C50 N100 w              24              34              50              33    #
   T3 C50 N50 w              39              65             100              38    #
   T3 C50 N75 w              41              52              74              61    #
-------------------------------------------------------------------------------------
            100          2214.8            25.1         28044.4             1.2    #
              0           464.6             4.8          2429.9             0.1    #

Ground:


 Techniques         -effort         -months        -defects          -risks    #
           T0 m              17              64               6              32    #
 T3 C100 N100 m              11              34               3               3    #
  T3 C100 N50 m              18              60               9               7    #
  T3 C100 N75 m              18              47               5               5    #
  T3 C25 N100 m               0              11               0               0    #
   T3 C25 N50 m               4              20               2               0    #
   T3 C25 N75 m               2              15               2               1    #
  T3 C50 N100 m               5              23               2               0    #
   T3 C50 N50 m              11              44               5               3    #
   T3 C50 N75 m               7              28               2               3    #
-------------------------------------------------------------------------------------
           T0 q              12              14               9              43    #
 T3 C100 N100 q              11               7               5               8    #
  T3 C100 N50 q              17              14              12              15    #
  T3 C100 N75 q              17              11               6              10    #
  T3 C25 N100 q               0               0               1               3    #
   T3 C25 N50 q               4               3               3               5    #
   T3 C25 N75 q               2               1               2               4    #
  T3 C50 N100 q               5               4               3               6    #
   T3 C50 N50 q              13              11               8              10    #
   T3 C50 N75 q               7               5               3               7    #
-------------------------------------------------------------------------------------
           T0 w              82             100              63             100    #
 T3 C100 N100 w              82              60              47              39    #
  T3 C100 N50 w              85              99             100              70    #
  T3 C100 N75 w             100              80              86              51    #
  T3 C25 N100 w              26              22              12              14    #
   T3 C25 N50 w              39              35              31              11    #
   T3 C25 N75 w              35              28              24              20    #
  T3 C50 N100 w              54              44              32              22    #
   T3 C50 N50 w              77              78              58              39    #
   T3 C50 N75 w              56              48              31              32    #
-------------------------------------------------------------------------------------
            100          1512.1            19.5         13341.2             1.0    #
              0           340.6             3.9          1505.5             0.1    #

Osp

Techniques         -effort         -months        -defects          -risks    #
           T0 m              54              82              22              49    #
 T3 C100 N100 m              11              30              10              10    #
  T3 C100 N50 m              12              40               7              24    #
  T3 C100 N75 m              16              58               8              25    #
  T3 C25 N100 m               0              11               0               2    #
   T3 C25 N50 m               7              26               5              16    #
   T3 C25 N75 m               7              19               4               8    #
  T3 C50 N100 m               8              25               4               8    #
   T3 C50 N50 m              12              42               7              27    #
   T3 C50 N75 m               7              29               5              14    #
-------------------------------------------------------------------------------------
           T0 q              16               7              11              35    #
 T3 C100 N100 q               8               5              12               4    #
  T3 C100 N50 q              10               9              10              19    #
  T3 C100 N75 q              16              15              11              13    #
  T3 C25 N100 q               0               0               1               0    #
   T3 C25 N50 q               6               5               7              12    #
   T3 C25 N75 q               5               3               6               2    #
  T3 C50 N100 q               7               4               6               3    #
   T3 C50 N50 q              13              11               9              19    #
   T3 C50 N75 q               6               5               7               5    #
-------------------------------------------------------------------------------------
           T0 w              93             100              56             100    #
 T3 C100 N100 w              63              51             100              27    #
  T3 C100 N50 w              62              66              81              62    #
  T3 C100 N75 w             100              97              89              59    #
  T3 C25 N100 w              21              21              24              10    #
   T3 C25 N50 w              48              46              65              42    #
   T3 C25 N75 w              48              35              61              20    #
  T3 C50 N100 w              57              44              55              22    #
   T3 C50 N50 w              73              71              82              65    #
   T3 C50 N75 w              53              49              65              34    #
-------------------------------------------------------------------------------------
            100           976.0            14.9         15980.1             2.1    #
              0           199.0             2.7          2022.7             0.5    #

Osp2:

Techniques         -effort         -months        -defects          -risks    #
           T0 m              50              82               8              16    #
 T3 C100 N100 m               9              41               5               8    #
  T3 C100 N50 m              15              60               6              17    #
  T3 C100 N75 m              18              52               6              15    #
  T3 C25 N100 m               1              16               0               1    #
   T3 C25 N50 m               7              36               4               7    #
   T3 C25 N75 m               3              22               0               0    #
  T3 C50 N100 m               4              24               1              10    #
   T3 C50 N50 m              12              47               5              11    #
   T3 C50 N75 m               9              35               3               9    #
-------------------------------------------------------------------------------------
           T0 q               9               7               2              34    #
 T3 C100 N100 q               8               7               8              20    #
  T3 C100 N50 q              13              12               9              45    #
  T3 C100 N75 q              16              11               8              29    #
  T3 C25 N100 q               0               0               1               5    #
   T3 C25 N50 q               6               6               6              21    #
   T3 C25 N75 q               2               2               1               9    #
  T3 C50 N100 q               3               2               2               9    #
   T3 C50 N50 q              11               9               7              30    #
   T3 C50 N75 q               8               6               4              19    #
-------------------------------------------------------------------------------------
           T0 w              88             100              21              54    #
 T3 C100 N100 w              60              67              80              71    #
  T3 C100 N50 w              82             100             100             100    #
  T3 C100 N75 w             100              88              88              79    #
  T3 C25 N100 w              30              29              28              28    #
   T3 C25 N50 w              59              60              55              59    #
   T3 C25 N75 w              39              39              28              34    #
  T3 C50 N100 w              39              39              36              57    #
   T3 C50 N50 w              74              78              72              76    #
   T3 C50 N75 w              70              61              56              58    #
-------------------------------------------------------------------------------------
            100           862.2            18.1          7153.8             1.2    #
              0           172.9             3.2           825.3             0.3    #

All:

Techniques         -effort         -months        -defects          -risks    #
           T0 m              13              37               7              27    #
 T3 C100 N100 m               7              26               3               0    #
  T3 C100 N50 m              14              46               7               3    #
  T3 C100 N75 m              10              35               6               1    #
  T3 C25 N100 m               0               7               0               0    #
   T3 C25 N50 m               5              18               4               2    #
   T3 C25 N75 m               2              12               1               1    #
  T3 C50 N100 m               4              17               1               0    #
   T3 C50 N50 m               8              26               5               3    #
   T3 C50 N75 m               6              24               4               1    #
-------------------------------------------------------------------------------------
           T0 q              12              13              10              35    #
 T3 C100 N100 q               8               7               5               5    #
  T3 C100 N50 q              17              17              10               8    #
  T3 C100 N75 q              10              11               8               7    #
  T3 C25 N100 q               0               0               1               2    #
   T3 C25 N50 q               4               3               5               4    #
   T3 C25 N75 q               2               1               2               3    #
  T3 C50 N100 q               4               4               3               3    #
   T3 C50 N50 q               8               7               7               7    #
   T3 C50 N75 q               6               6               7               4    #
-------------------------------------------------------------------------------------
           T0 w              65              70              65             100    #
 T3 C100 N100 w              66              56              50              20    #
  T3 C100 N50 w             100             100              84              31    #
  T3 C100 N75 w              96              83             100              28    #
  T3 C25 N100 w              17              17              15               8    #
   T3 C25 N50 w              43              38              48              12    #
   T3 C25 N75 w              27              26              26              12    #
  T3 C50 N100 w              51              37              29              15    #
   T3 C50 N50 w              70              57              54              21    #
   T3 C50 N75 w              71              53              51              20    #
-------------------------------------------------------------------------------------
            100          3002.5            35.6         20668.2             1.2    #
              0           400.5             6.2          2341.5             0.1    #


Thursday, May 8, 2014

Pantry and Survey Correlations

Initial post 5/08: we have some charts, but outlier removal is needed.

Items A:
Trans Fat
Sodium
Sugars
Calories
Saturated Fat
Total Fat
Cholesterol
Total Carbohydrate
Protein
Dietary Fiber


Items B: 
cholesterol_mg_usda
protein_g_usda
total_fat_g_usda
total_sugars_g_usda
alcohol_g_usda
total_monounsaturated_fatty_acid
total_saturated_fatty_acids_g_us
energy_kcal_usda
dietary_fiber_g_usda
total_polyunsaturated_fatty_acid
carbohydrate_g_usda








JPL prelim results for may


Val0

How?

ONLY on new data:

1. Do a LOO experiment. At each instance gives, left out row and rest rows.
2. Cluster rest rows.
3. Find nearest cluster to left out row with nearest attribute distance of centroids to left out row.
4. Estimate = Mean or median of LogicalEKLOC/ Logical Delivered,
 Actual = Actual LogicalEKLOC/Logical Delivered.
5. Calculate MRE.


Val1

Val2

How?

 0) cluster the dense columns 
    1)  take the "left out" example,  described in terms of:
                a) the new columns ("new" means "not traditional cocomo")
                b) its "class" e.g. "flight systems"
                c) act effort 
    2)  find the "left out" example's cluster and extract the LOC in that cluster
               - note: until step 5 we will ignore the actual effort
    3)  go to darren's rig and generate effort predictions (constrained by (2) and (1b))
    4) show the 50 to 70th percentile range of (3)
    5) mark on that  range the actual effort
    6) dance and sing if the the actual effort (from 1c) is in the range of (5)

Estimate = Mean/Median of generated effort from darren's rig
Actual = actual effort of project



The 50 - 70 range sheet:

http://unbox.org/things/var/nave/lpj/out/master/Effort_check_50_70.xlsx



Monday, May 5, 2014

Magic Learner-JPL and others

FOR JPL


Study 1:

  1. Predict LogicalDelivered, excluding EKLOC, inheritance.
  2. Predict LogicalEKLOC, excluding delivered, and including inheritance.
Study 2:
  1. Boring COCOMO using 1.1
  2. Boring COCOMO using 1.2

Boring COCOMO ready and set to go.

Effort Predictions(val2):


so, we need a LOO experiment where we:
    0) cluster the dense columns 
    1)  take the "left out" example,  described in terms of:
                a) the new columns ("new" means "not traditional cocomo")
                b) its "class" e.g. "flight systems"
                c) act effort 
    2)  find the "left out" example's cluster and extract the LOC in that cluster
               - note: until step 5 we will ignore the actual effort
    3)  go to darren's rig and generate effort predictions (constrained by (2) and (1b))
    4) show the 50 to 70th percentile range of (3)
    5) mark on that  range the actual effort
    6) dance and sing if the the actual effort (from 1c) is in the range of (5


http://unbox.org/things/var/nave/lpj/out/master/effort_check_with_cocomo_50_70.xlsx


Graphs learnt on whole *old+new* with specific conditions(val0):




OTHER THAN JPL

Results for tables showing different techniques:

http://unbox.org/things/var/nave/lpj/out/master/*preptab*.dat

Contrast sets sorted:

http://unbox.org/things/var/nave/lpj/out/master/*contrast_sets_sorted.dat

Trees for different models:

http://unbox.org/things/var/nave/lpj/out/master/*dtree*.dat

Other graphs:


MRE with and without learning 20 records around cluster centroid: