Big Data for Development: Challenges & Opportunities

Innovations in technology and greater affordability of digital devices have presided over today’s Age of Big Data, an umbrella term for the explosion in the quantity and diversity of high frequency digital data. These data hold the potential as yet largely untapped to allow decision makers to track development progress, improve social protection, and understand where existing policies and programmes […]

Continue reading »

Data Streaming in Hadoop: a Study of Real Time Data Pipeline Integration Between Hadoop Environments and External Systems

The field of distributed computing is growing and quickly becoming a natural part of large as well as smaller enterprises’ IT processes. Driving the progress is the cost effectiveness of distributed systems compared to centralized options, the physical limitations of single machines and reliability concerns. There are frameworks within the field which aims to create a standardized platform to facilitate […]

Continue reading »

Reusing Results in Big Data Frameworks

Big Data analysis has been a very hot and active research during the past few years. It is getting hard to efficiently execute data analysis task with traditional data warehouse solutions. Parallel processing platforms and parallel dataflow systems running on top of them are increasingly popular. They have greatly improved the throughput of data analysis tasks. The trade-off is the […]

Continue reading »