From conventional to search-focused programming environments

Abstract

Conventional programming environments operate with a complex interplay among different tools for accessing different resources related to source code. These resources contain various metadata, such as code dependencies, naming organization or version control. Each kind of metadata carries its specific format, such as binary or semi-structured files of various formats, and corresponding access methods.

Code entities –- such as functions, constants or variables –- do not have globally unique persistent identifiers in conventional programming environments. Access to external code is name-addressable and resembles hierarchical file systems. User-chosen hierarchical names, however, are not globally unique. Name conflicts may, therefore, arise among definitions from different libraries or even the same library of different versions. The lack of persistent globally unique identifiers prevents code entities from being directly associated with some metadata. The non-unique identifiers depend on an implicit context: user-chosen identifiers may refer to different entities at different times or locations. Additionally, conventional programming environments specify project-level dependencies in which an unused code may need to be included when dependencies are resolved.

To address these issues, this thesis proposes a design for programming environments, named Search-focused Programming. Search-focu\-sed Programming environments have three key properties. First, code entities are deterministically assigned globally unique persistent identifiers based on their content. With unique identifiers, relations can associate dependencies among code entities directly. Second, code and related metadata are uniformly stored. The uniformly stored data can be accessed in two ways: by retrieving data using a declarative query language, and by reading and writing individual records in the transaction log. Third, the shared source code is separate from the locally edited code. Two proof-of-concept prototypes demonstrate two different directions in which one can interact with code. The first prototype presents a command-line interface, providing indirect interactions similar to conventional programming environments. The interaction is realized through bulk processing local source code files and generating corresponding queries or transactions. The second direction is in the form of a web-based interactive interface. The interaction happens by directly looking up individual code entities and manipulating them in a structure-driven editor, i.e. without the bulk processing of layers of naming structures. Finally, this thesis examines the effects of definition-level dependency tracking on existing source code bases. A popular community project repository was analysed, and the source code of the top five most depended on projects was found to never be fully utilized by libraries that declared their dependencies on them. A substantial fraction of the code entities in these five projects remained in more than one version of their respective libraries. Lastly, overhead was measured in compilation speed on client code due to unused code, which would be removed with definition-level dependencies. These results suggest the possible merits of employing the ideas from Search-focused Programming in existing programming environment infrastructures.