INESC TEC starts European project on Big Data

In February 2014, the High-Assurance Software Laboratory (HASLab) and the Centre for Information Systems and Computer Graphics (CSIG) of INESC TEC started working on the European project LeanBigData (Ultra-Scalable and Ultra-Efficient Integrated and Visual Big Data Analytics). The main goal is to solve three great challenges of analytical processing in Big Data: the cost, in terms of resources, of scaling big data analytics for streaming and static data sources; the lack of integration of the existing technologies to manipulate Big Data and high response times; and the insufficient end-user support leading to extremely lengthy big data analysis cycles.

The goal with this project is to create and develop three efficient systems to manipulate Big Data: a novel transactional NoSQL key-value data store, a distributed complex event processing (CEP) system, and a distributed SQL query engine. In the same line of thought, the research team, which includes HASLab researchers Rui Oliveira, José Orlando Pereira, Francisco Moura, José Nuno de Oliveira and António Ribeiro, and CSIG researcher Alexandre Carvalho, will create an integrated platform with the three main technologies for Big Data”, NoSQL, SQL and Streaming/CEP, that will improve response times for unified analytics over multiple sources and large amounts of data, avoiding the inefficiencies and delays introduced by existing extract-transfer-load (ETL) approaches.

The project should also support an end-to-end Big Data analysis, eliminating the four main sources of delay in the analysis of data, using automated discovery of anomalies and root cause analysis, incremental visualisation of long analytical queries, drag-and-drop declarative composition of visualisations, and efficient manipulation of visualisations through hand gestures over 3D/holographic views.

Finally, the LeanBigData will demonstrate these results in a cluster with 1,000 cores in four real industrial use cases with real data, paving the way for deployment in the context of realistic business processes.

Other than INESC TEC, the project Consortium includes the company Portugal Telecom, the Universidad Politecnica de Madrid and the companies Computer Associates and ATOS (Spain), Intel (Israel), the Foundation for Research and Technology (Greece), the Institute of Communication and Computer Systems (United Kingdom), and SyncLab SRL (Italy). The project has a duration of 36 months and it has received funding from the European Union’s Seventh Framework-Programme.