LargeCollections - A fast and persistent cache

with a java.util.Map interface

This week Axiomine released LargeCollections as open-source under the Apache License.

LargeCollections supports java.util.Map implementation which is backed by LevelDB. This allows your collections to grow very large as it does not use the JVM heap memory.

The primary purpose behind creating LargeCollections was to support java.util.Map. . . .

Read More

January 27, 2015

The MapReduce Itch

all the NextGen frameworks are scratching

MapReduce has an inherent and intrinsic limitation built into its architecture which impacts throughput. All new frameworks are trying to address this limitation in some form or the other.

MapReduce Limitations in Hadoop

Some of the characteristics of MapReduce which impose serious throughput constraints in practice are

  1. Starting a MR Job . . .

Read More

December 30, 2014

MapReduce Internals

A MapReduce program is composed of four program components

  • Mapper
  • Partitioner
  • Combiner
  • Reducer

These components execute in a distributed environment in multiple JVM’s. The two JVMs important to a MapReduce developer are the JVM which executes a Mapper and that which executes the Reducer instance. Every component you develop will execute in . . .

Read More

December 09, 2014