Friday, January 28, 2011

Open Issues in Empirical SE

Q1: Generality of the Microsoft conclusions?

Q2: How to support the following kinds of business intelligence?


Other approaches to business intelligence:

Talk at Microsoft Research, Jan 27, 2011

Talk at Microsoft Research, Jan 27, 2011

Slides (7MB) :

Download (250MB) :

The rapid pace of software development innovation challenges empirical software research to keep up, if it is to deliver actionable and useful results to practitioners. The empirical software engineering research field has not always been able to deliver this. Recently, it has become increasingly apparent that rigorous data collection and analysis can be so expensive and time-consuming that empirical software engineering studies, which seek to understand the costs and benefits of software development solutions in practice, greatly lag the pace of innovation in the field. In too many cases, a trusted body of empirical results can only be built up after the innovative solutions that they are studying are already well on their way to obsolescence or standard practice. However, we argue that recent advances put a sustainable and increased research pace within our reach. A suitably scaled-up and nimble empirical research approach must be based upon the “crowd-sourcing” of tough empirical problems. 

Tuesday, January 25, 2011

Will McBurney

Currently a graduate student in Computer Science at West Virginia University, Will McBurney is a programmer, musician, and blogger on the side. Will McBurney began programming in his Senior year of High School in Charleston, WV, his hometown. Originally attending WVU as an undergraduate aspiring to become a Mechanical Engineer, Will switched majors midway through his freshmen year to Computer Science.

It is not secret Will is a major nerd. Will skipped his Senior Prom to go to National Science Bowl in 2006 and has read more Star Wars books than most people have read at all. On the inter-tubes, Will uses the psuedonym "Death by Smiley", a name that was made up spur of the moment while playing Halo in the dorms at WVU.

-Will McBurney

Active Learning

Kel w/ Cliff:

Andrew w/ Heuristic Randomness:

Rank Deltas for COMBA

Which features define CC?

Division of available datasets into subsets according to different criteria (language, hardware, resource etc.) may define closeness of a cross-company setting. The results and subsets are here.

Bias and Varince of Testing Strategies on Effort Datasets

A short report is here.

Monday, January 24, 2011

Another look at Fayola's data

map-reduce & Hadoop


MapReduce is a patented[1] software framework introduced by Google to support distributed computing on large data sets on clusters of computers.[2]
MapReduce libraries have been written in C++C#ErlangJavaOcamlPythonRubyF#R and other programming languages.

Material for interactive new media design

popularity is (almost) a perfect predictor for defects

One surprising observation from the Helix study [50] was that(a) most classes are not popular and (b) patterns of the most pop-ular classes emerged very early in the lifetime of an open-sourceproject. Specifically, popularity and age of a class maintain a pos-itive monotonic relationship throughout the lifetime of a system[50], that is, as a system matures, popular classes tend to becomeeven more popular. 

This contradicts our reading of Fowler [20] andBeck’s work [6] that object-oriented quality assurance requires in-volved constant refactoring (e.g., Beck’s TDD loop of write tests,run test, refactor). We cannot see evidence of constant, widespreadrefactoring in open-source projects studied by Helix [50].

The implications of this lack-of-refactoring for the theory of object-oriented development is discussed elsewhere [50]. This paperfocuses of class popularity since (a) it is a stable concept for thelifetime of an object-oriented system and (b) popularity can lead todefects via:

  • Defect injection:As developers work with the popular classes, they make the occasional mistake. Some of these mistakesresult in code defects. Since developers work on popular classes more than other classes, then most developer defects accumulate in the popular classes.
  • Defect discovery: Since developers work mostly on popular classes, they are most likely to uncover those classes’ defects.
We demonstrate that in 33 open-source Java projects popularity-based defect predictors work within 4% of a theoreti-cal upper bound on predictor performance (this is the basis for ourclaim that such predictors are “nearly perfect”).  

empirical SE v2.0

timm, trying to map out a research agenda for the next decade

More on story content generation

Tuesday, January 18, 2011

What makes a game fun

Hello, hi,

My name is Joe and I'm a new PhD student at WVU, got my MS in CS just last year, and working towards the big goal with a vague idea and a genuine interest for AI in games. I'm hard of hearing, and generally have a tough time understanding people in a group setting, but usually if there's only one person speaking -towards- me, with a good voice, then it's fine. (Sorry if I seem confused sometimes, usually trying to figure out what you just said.)

Last semester I worked on the idea of "What makes a game Fun?" and found myself digging through a lot of unofficial resources towards concepts in a game that heighten the level of enjoyment had by players. I wrote a survey-type paper and outlined what I thought were the core values of "Fun in video games." Those four were Gameplay, Originality, Replayability and Story (GORS). Replayability is key; any time a player picks up a controller, there is a drawing factor that keeps them attracted to the game and continuously replaying. Once that allure is lost, replayability is lost, and the game no longer has potential to provide fun (at least for a while, but there's always a chance the allure returns at some later point - yet another key point to replayabilty.)

Anyway, second half of semester I put the four dimensions of fun together in a game which the user could play and "set" the levels of each dimension. i.e., the user would set originality to be of "low" setting, making item names and enemies sound very generic, as opposed to original and unique. Called Cre-Ativity, the game plays like Zelda, in a top-down fashion developed with OpenGL in C++. Players can use one of four weapons; acquire armor, gain experience, become stronger and bash enemies while exploring in aims of completing a general story-driven quest.

At this point, the semester was over and one of the key questions that remained was; "How could you develop story content according to user-input?" More on that topic to come.


Cross vs. Within Company Experiments

Tentative results are here.
Selected cross-company datasets are given below..
finnishAppArea = {'finnishAppArea2','finnishAppArea7','finnishAppArea10'};
finnishAppType = {'finnishAppType1','finnishAppType2345'};
kemererHardware = {'kemererHardware1','kemererHardware23456'};
maxwellAppType = {'maxwellAppType1','maxwellAppType2','maxwellAppType3'};
maxwellHardware = {'maxwellHardware2','maxwellHardware3','maxwellHardware5'};
maxwellSource = {'maxwellSource1','maxwellSource2'};
chinaResource = {'chinaResource1','chinaResource2','chinaResource3','chinaResource4'}

Is 1*LOO ok or do I need 20*LOO?
Is using only static-k better? 

Testing using only the X dimension in IDEA

New results after discretizing with Fayyad Irani

This week...

Tim has written two subjects

Learned about map-reduce and clojure and wondered about applying it to WHICH.

Coded up idea, knn, cross-val, and run lots of experiments

Worked with physics on an NSF proposal

Planned next week's talk at microsoft, redmond

Worked with Martin Shepperd on a journal special issue.

Worked with Ayse Bener on a journal special issue

And written a research paper

Over to you....

Tuesday, January 11, 2011