Thursday, March 16, 2017

Improving io-stats translator

Hey all,

My Outreachy internship primarily involved tasks related to improving the io-stats translator of Gluster.

So what is a translator ?

I have shared a video in my last post regarding "Hacking Gluster FS" in which they explain it clearly. But to put in brief, The translators convert the requests from users into requests for storage. It is layered in such a way that the request passes through them. A translator can also modify requests on the way through.

The io-stats translator on which my tasks were related to is used to instrument the Gluster from within. The volume profile command provides an interface to get the per-brick I/O information for each File Operation (FOP) of a volume. The per brick information helps in identifying bottlenecks in the storage system.

To add a brief on my tasks,

1) As stated above, the profile command provides an interface to get the per-brick IO information for each file operation of a volume. So it is necessary that all file operation are listened for. To achieve this, my first task involved in finding the missing file operations and adding it to the io-stats translator.

The commit related to it can be found here.

2) While doing the above, my mentor suggested that doing the above using a code-generation method will help in reducing the code in Gluster code as many lines of code for each file operation looked similar and can be generated by means of a common template.

An initial step towards achieving this is to have all details related to every file operations together. In this attempt, we found a file operation in the name of "compound" is missing in the list. We attempted to add it. The related commit can be found here.

3)With the support to all the file operations added, our next step was to make the code generation framework work. The related work can be found here. We had few issues with it and the same can be found in the comments of the commit.

With this completed, my next tasks involve in making the Gluster profile command more useful. I shall share my learning and work related to it in my next blog post.

Tuesday, December 6, 2016

Outreachy, Getting started

Hello all,

I'm Menaka M from Tamil Nadu, India. You all must be knowing what Outreachy is and only so you're here ;) I got a chance to be the part of this wonderful program under Gluster organisation for this round.

Let me go through the application process that I followed for getting into this program in brief.

A little about me:

I got to know about Outreachy through one of my friend who participated in GSoC earlier. By then, I have completed my B.Tech in Information Technology. Sadly, I was not allowed to go for a job as it was too far from my hometown.

Outreachy looked like an awesome opportunity if I can get a chance to take part in it. I should admit that I am not a geek. But when provided a chance to learn, I was more than enthusiastic to learn new things under someone's guidance.

After going through the projects page, I initially spent some quality time in learning about Gluster and also in setting up my development environment.

I introduced myself to the community and started working on the tasks given in the page.

Whenever I got stuck, I learned more about that and posted questions with my leanings attached to it.As I have told you, I was very much new to participate in Open-Source community forums. But the help that I received from the community members were very much motivating to learn more and try again and again to complete the tasks. I did not have anyone else other than the Gluster community members to take help from.

I should thank them a lot for helping me throughout the application period and motivating me to apply. A wonderful community for a fresher to get involved and take help!

And to continue, a little on more about Gluster.

So, what is Gluster ?

The Gluster File System (GlusterFS) is an open source distributed file system that can scale out in building-block fashion to store multiple petabytes of data. It would be very much interesting to learn more about it. Go for it :)

So, What are the tasks for this internship that are related to Gluster ?

My project was to introduce instrumentation tooling into Gluster. Later, it took different paths but the motto is still the same.

How did I learn about Gluster ?

I searched for so many online resources to learn about Gluster and also took much help from the docs available in their official site.

One such good resource that helped me is Hacking GlusterFS

Though it is an old video, it helped me a lot to understand the basics of it.