Two student assistant vacancies – Lighthouse project
We are looking for two student assistants (paid job, 1 day a week) for the Lighthouse project: large-scale graph analytics on Giraph
Supervisors: Peter Boncz (Large Scale Analytical Data Management, VU and CWI), Spyros Voulgaris (Computer Systems, VU).
Data-driven processes provide insightful views into data and its dynamics. In recent years, a number of general-purpose platforms, such as the Hadoop ecosystem, have been proposed to analyze large datasets. Graph processing is gaining momentum in this landscape, due to the always increasing number of relevant datasets expressed through graph data models, such as Social Networks, the Semantic Web, the Internet, problems in the domain of bioinformatics, transportation networks, etc. In these domains, analytics comprise of graph traversals and aggregations over values computed for vertices and edges.
Pregel and its open-source counterpart Giraph have emerged as leading platforms for large-scale graph processing. These platforms are able to run graph computations on graphs with up to trillions of edges. However, writing algorithms directly in the Pregel/Giraph API is error-prone and time-consuming. High-level languages, like Cypher are currently being used for querying graph databases. However, these databases do not scale to graphs in the order of trillions of edges.
We have developed Lighthouse, a graph-processing engine where computations are expressed through a high-level language, and are executed by optimized Pregel jobs. In particular, we have implemented a query execution engine for Cypher on top of Giraph. Lighthouse can run Cypher queries on millions of vertices within minutes, but more work needs to be done. For this project, we aim at tackling the following challenges:
1.Aggregations: Lighthouse can now perform pattern matching on huge graphs, finding sub-graphs with particular properties and topologies. We want to be able to compute aggregations on these sub-graphs.
2.Path queries: Simple pattern-matching is powerful, but we want to enable Lighthouse users to express complex patterns through paths, like shortest paths and path-expressions.
3.Higher scalability: Extend Lighthouse
The Institute for Advanced Study (IAS) and Data Science Center (DSC) of the University of Amsterdam (UvA) are joining forces in a new joint fellowship for (teams of) researchers interested in exploring the intersection of data science and other disciplines. The first fellow of the new program, Dr. Davide Ceolin, will start his fellowship at the IAS on June 17. Ceolin is currently affiliated with CWI.
Nanda Piersma, lecturer at the University of Applied Sciences (AUAS), has been appointed as a crown member of the Social and Economic Counsil (SER). This is the first time that a lecturer has been appointed as member of the SER. Nanda Piersma was approached because of her expertise in the field of digitisation.
The ADS Thesis Awards aim to promote excellence in Data Science and AI from students at BSc and Master level. The Awards are open to students from all Amsterdam-based knowledge institutes.