Distributed storage systems for big data

Project name

Distributed storage systems for big data

Project description

The group maintains a framework called dmlite which is used to integrate various types of storage with different protocol frontends. It is the basis of a number of the group’s products such as the Disk Pool Manager (DPM), a grid storage system which holds over 50PB of storage in the global infrastructure. DPM/dmlite extensions
The task is to contribute to the dmlite project by working on functional extensions to the framework. Example projects include

  • Exposing system data through a “procfs” style plugin
  • Incorporation of new AA mechanisms, eg outh
  • Creation of a web admin interface
  • Work on draining and file placement within the system
  • dmliteSE

Help to realise the group's vision of a “dmliteSE” by working on the gradual retirement of legacy daemons within the DPM system. In this context, tackle the modernisation of pool management and file placement, and the incorporation of new resource types (eg cluster file systems) into the system. Complete the functional development required to allow operation of a disk storage system purely through standard protocols.

Required skills

C++/Linux

Learning experience

This project offers the chance to become involved with one of the storage systems used in computing for LHC and will give an opp

Project duration

3 to 9 months depending on the selected task

Project area

Data Management

Contact for further details

oliver.keeble@cern.ch

References

https://svnweb.cern.ch/trac/lcgdm/wiki/Dpm

CERN group

IT-SDC

Status

Submitted Submitted by Catharine Noble on Friday, January 15, 2016 - 12:00.
Friday, January 15, 2016 - 12:00