Scaleable Code Clone Detection

Code clone detection helps connect developers across projects, if we do it on a large scale. The cornerstones that allow clone detection to work on a large scale are: (1) bad hashing (2) lightweight parsing using regular expressions and (3) MapReduce pipelines. Bad hashing means to determine whether or not two artifacts are similar by checking whether their hashes are identical. We show a bad hashing scheme that works well on source code. Lightweight parsing using regular expressions is our technique of obtaining entire parse trees from regular expressions, robustly and efficiently. We detail the algorithm and implementation of one such regular expression engine. MapReduce pipelines are a way of expressing a computation such that it can automatically and simply be parallelized. We detail the design and implementation of one such MapReduce pipeline that is efficient and debuggable. We show a clone detector that combines these cornerstones to detect code clones across all projects, across all versions of each project.

Quantitative assessment of sense and antisense transcripts from genes involved in antigenic variation (vsp genes) and encystation (cwp 1 gene) of Giardia lamblia clone GS/M-83-H7.

Antigenic variation of the intestinal protozoan parasite Giardia lamblia is caused by an exchange of the parasite's variant surface protein (VSP) coat. Many investigations on antigenic variation were performed with G. lamblia clone GS/M-83-H7 which produces surface antigen VSP H7. To generate novel information on giardial vsp gene transcription, vsp RNA levels were assessed by quantitative reverse transcription-(RT)-PCR in both axenic VSP H7-type trophozoites and subvariants obtained after negative selection of GS/M-83-H7 trophozoites by treatment with a cytotoxic, VSP H7-specific monoclonal antibody. Our investigation was not restricted to the assessment of the sense vsp transcript levels but also included an approach aimed at the detection of complementary antisense vsp transcripts within the two trophozoite populations. We found that sense vsp H7 RNA predominated in VSP H7-type trophozoites while sense RNA from only one (vsp IVg) of 8 subvariant vsp genes totally analysed predominated in subvariant-type trophozoites. Interestingly, the two trophozoite populations exhibited a similar relative distribution regarding the vsp H7 and vsp IVg antisense RNA molecules. An analogous sense versus antisense RNA pattern was also observed when the transcripts of gene cwp 1 (encoding cyst wall protein 1) were investigated. Here, both types of RNA molecules only appeared after cwp 1 had been induced through in vitro encystation of the parasite. These findings for the first time demonstrated that giardial antisense RNA production did not occur in a constitutive manner but was directly linked to complementary sense RNA production after activation of the respective gene systems.