Wednesday, August 26, 2015

Virtue of Necessity. Canary, sublime your company.

The past July 16th I participated in the Tenerife LAN Party, in its section Tenerife Innova, invited by the Free Software Office of La Laguna University, included in the track titled (free translation from Spanish) Open Source from the Canary Islands, stories told in First Person.

This Free Software Office is well known in Spain for managing the biggest KDE deployment in Spain with 3k computers spread in several computer labs, laboratories and libraries, among other internal projects.

My talk (20 min) had as title something like: Virtue of Necessity. Canary, sublime your company. You can find the slides (in Spanish) in my site or in my Slideshare account.

What was the talk about?

The natural growth path for a software company is creating a site, grow until it reaches a point in which, a second production site is needed. Meanwhile, departments like sales and technical support might grow distributed. As the company grows, the number of production sites grow  with it. The organization structure vary with the nature of the company, sometime production teams are replicated across different sites, sometimes different business units are divided per site and others a particular site host teams that take care of different products-(micro)services.

In general, if the company follows an "agile" approach, it will try to reduce the inter-sites communication needs by placing the team members together in a particular site. Based on my experience and how FOSS has been developed, depending on the market you are playing, turning your company into a distributed environment might be a smart move. 

Let's start providing some context and definitions.

What do I mean by sublimation in this context?

Sublimation is a state change in which a substance transit from the solid state to the gas state without going through the liquid one.

What do I mean by a distributed environment?

In most agile literature, in fact in most software development management books, distributed environments are multi-site distribution. But in Open Source, we refer to a different environment. I have came out with the following (subjective) definition:

A distributed environment/organization:
  • Has no site with a significant number of employees (developers). Most of them work remotely.
  • Access most (if not all) corporate application though web, not through VPN, that is, are WAN environments, not LAN.
  • Uses chat-IRC (or equivalent) and video conferences as the default synchronous communication channel.
  • Employees spread across several time zones.
  • Multicultural environment, having English as the default language.
  • Organize regular face to face sprints (as we understand it in FOSS=hackathons), maybe even a corporate event where the whole company or business unit get together.

Open Source as distributed environment

Free/Open Source Software has a geographically distributed nature. As you know, most of the relevant communities, no matter which size are we talking about, are formed by developers located in many different countries, working from home or a company site. If we take a look at the most relevant ones, they are truly global. Every tool, every process, has been designed (intentionally or not) having in mind this distributed nature.

Now that Open Source is everywhere, more and more companies are embracing it, participating on its development, collaborating in global communities. from the process perspective, they are being influenced by the Open Source way, including its distributed nature.

Many of the companies that are embracing Open Source are understanding that adapting to this new environment makes collaboration easier. It reduces the friction between community and internal processes. There is a long way to go but I believe it is unstoppable for a variety of reasons (out of the scope of the talk). We are starting to see more and more organization that are fully distributed and start-ups that are born with such structure in mind.

Canary Islands, a fragmented market.

The Canary Islands is a group of 7 islands in the Atlantic sea, with two major ones (900k people each) and 5 smaller, for a total of 2.1 million people and 11 million tourists per year. Obviously tourism is the main industry, so there are 6 international airports, two national ones and 10 harbours, half of which regularly receive big ships/cruises.

Data connectivity has improved a lot the last 10 years but, due to the difficult geography, it is unequally distributed across the islands. Even within the main islands, Tenerife and Gran Canaria, there is a significant percentage of surface with zero internet coverage.

So it is a very fragmented market and, although with first class communication infrastructure, travel across the islands takes a significant amount of time, it is expensive and connectivity might be a challenge. In general, the transportation strategy has been designed to bring people from Europe not for internal mobility.

This means that, as a software company, consolidation/growth in such market is tough, very tough, even if you focus on tourism.

Software companies there expand following the "natural" approach, which is by creating a software production centre in one of the main islands, providing support from there to the other islands. Until you consolidate your position, software companies cannot afford to have developers/technical support in the second main island. If service/support is required in one of the small islands, you simply travel there. The limitations that software companies has to face due to the market conditions, rarely allow you to create a second software development centre in the Canary Islands.

There are very few Spanish cities with daily direct planes from/to Canary Islands throughout the year. Madrid and Barcelona are the biggest markets but also the most expensive cities. The flight takes 2:30 hours to Madrid and 3 hours to Barcelona, which is a lot for European standards. So opening a second development centre in the mainland keeping the headquarters in the Canary Islands is a real challenge.

In other words, if you want to scale your business, you need to assume bigger risks than companies based in the continent, despite being a cheaper place and having plenty of professional due to the existence of two Universities.

... but,

In my talk I tried to show that all those limitations can be turned into advantages if organizations, early in their consolidation process, or even from the very beginning, adopt a distributed approach. These constrains offer a first class laboratory to experiment with some of the key variables that need to be managed when scaling up your company, while leaving aside some of the most complicated ones, related to great extend with the internationalization of the organization.

I made a call to sublimate your company, going from an "on-site" to a fully distributed state, ignoring the multi-site state. Even better, create your software company as a distributed environment since the very beginning.

Why sublimating your company in the Canary Islands?

I summarized the advantages of sublimating your company if you are based in the Canary Islands, Spain, in the following statements:
  • Distributed environments adapt better to the Canary Islands environment.
  • It will prepare you earlier for the internationalization phase, keeping a smaller size.
  • Distributed environments adapt well to certain business models and support needs, that are becoming popular nowadays.
  • If your company want or is already heavily involved in Open Source communities, adapting your internal processes to those collaborative environment you participate on is easier.
  • Talent attraction is less difficult.
  • Some of your fixed costs turn into production costs, that is, you gain flexibility.

Which variables will be affected by subliming your company?

These are the most relevant variables to consider:
  • Cost per employee: reduction of fixed costs per employee. Increase of travel costs.
  • Organization chart: from vertical to horizontal
  • Human Resources policies: training, coaching, people management. From f2f to online.
  • Data privacy and security: adapt to a WAN environment.
  • Tools: adapted to distributed with higher latency environments
  • Schedule / availability: culture shift, from presence to availability
  • Development and support methodologies: from f2f conversations to remote synchronous/asynchronous channels. From agile to FOSS? Is FOSS agile?. From 8-10 hours or rotation to a higher daily production/availability window.
  • Costumers relation/engagement: from tradition account management to "community" engagement management. Engineers interface customers.
  • Transportation and connectivity: employees need to be connected and travelling demands will increase. Accountability/reimbursement processes will be more complex.
  • Business model: your business model might need to adapt to your new distributed nature.
  • Potential market: the presence of employees in new areas and the fact that your processes are adapted to distributed environments might change your target market and/or nature/size of potential customers.
  • Competitors: the influence of your new distributed nature might alter your positioning against competitors.

There are more but these ones are the ones that should be considered carefully before subliming your company in the Canary Islands. As you grow, internationalization will knock at your door very soon. There are other variables to consider in that case. They are not the scope of this talk:
  • Time zones
  • Multiculturalism
  • Language barrier
  • Retributions and incentives. Cost of living.
  • Fiscal and employment legislation differences
  • Travel and accommodation costs and reimbursements
  • Accountability and taxes
  • Many more...

Summary

The Canary Islands is a tougher market than the mainland of Spain. Adopting early in the software company life cycle a distributed nature allow you to adapt better and faster to this environment, preparing you better for later stages too, specially the internationalization phase. Sublimation provides you a competitive advantage, specially if you develop Open Source and participate in open collaborative environments.

There are a number of variables that should be carefully considered though. Managing them correctly is a requirement to succeed.

Friday, June 12, 2015

openSUSE transformation step 2. The user oriented distro.

openSUSE Tumbleweed is a rolling release that was designed with a clear goal, target and metric. It was developed following a clear picture of where to go. Design it was a painful but unavoidable process that challenged an assumption established in many people mindset back in 2012. Latest/greatest and stability were incompatible. Hence, Factory, the rolling release back then, was for hardcore SUSE/openSUSE OS developers only. As rolling distros, there were more popular and better options out there.

This summary might help you to get some background.

Goals are easier to achieve if you have a good reference to beat. For those who worked in the project, Gentoo and specially Arch Linux were those references. As you can imagine, transform openSUSE required management support. We had it, specially from Roland Haidl, Operations and Communities Director at SUSE back then. He created the environment that allowed those who worked in his department to be creative.... and take risks.

Simplifying, for the new "development version", a.k.a Tumbleweed (former Factory), the goal was to implement a model that allowed us to improve the existing Factory one, based on continuous delivery. The target chosen were our core contributors (packagers fundamentally) and the metric was, in summary, to make sure that, no matter how wrong things could go after an update, you would always have a console and network, so you would be able to revert your change. In terms of the process, the resulting integration deployment processes should be transparent, not just internally but also from our community members perspective. It also needed to be simpler in order to gain contributors, not just users. And it needed to empower them to own it.

Instead of following what SUSE was doing back then, the company dedicated resources to challenge itself. As result, openSUSE Tumbleweed is today, not just the best rolling distro out there, with all what that means in terms of excitement among its contributors, but is generating higher value to SUSE, since the company have an outstanding playground at home that allows them to incorporate true innovation into their production process before their competitors do.

openSUSE is discussing nowadays to take a second step, this time focused on its user oriented version. Today is openSUSE 13.2.

In my opinion, based on the previous experience, and independently of the decision/discussion process chosen, the same steps need to be taken. They are unavoidable in any transformation process. It is necessary to define a clear goal, something short that you can explain and understand easily, a clear target and a key metric that helps to clarify the "acceptance criteria" to be used during the whole process.

Like back then, I would like to see SUSE challenging itself, putting in question well established principles within the OS industry. Again, choosing a reference would make the final picture easier to achieve.

Most openSUSE users are desktop users and sysadmin. If, as I conclude from the latest oSC15 videos and factory mailing list discussions, sysadmins are the chosen target, It would be great to see SUSE/openSUSE challenging the assumption that, through a continuous delivery process, you cannot release a stable and high quality (for the target) distribution. That stability is only achievable through a waterfall like model. I would choose CoreOS as reference. It is a project that, based on different questions, is providing innovative answers to new challenges.

I would like to see that, base on the current process (standing on the shoulders of giants) openSUSE/SUSE creates a process that "pulverize" the current mindset, deprecating many of the existing problems, focusing on solving new ones. Imagine the best of both worlds, a new paradigm of OS with the green values.

It took about a year and a half for a dedicated team to release what today is Tumbleweed. I think that this second challenge is bigger than the first one. An even bigger commitment from SUSE will be needed in order to succeed.

But if the resources are there, the creative environment is set, the right steps are followed and the openSUSE community supports the effort, there is nothing that can stop the project to achieve what today are dreams. SUSE has the talent, and the experience, to make it happen.

I wish them all the best in this new challenge.

Monday, May 18, 2015

Bitacora: impact mapping

Introduction


This is the fourth of a series of articles about Bitacora. Please read the previous ones to get full context:
  1. The diary (aka bitácora): towards alignment in distributed environment.
  2. Bitacora: environment definition.
  3. Bitacora: personas 

Why Impact Mapping?

The steps taken so far are standard for me since long time ago. At this point though, depending on the nature of the product, I did not have a fixed procedure to follow until the creation of user stories. At some point I ended up creating use cases and relating them through mind maps.

A few months ago I read Impact Mapping, from Gojko Adzic, a reputed Agile consultant and writer. He proposes a method to go from business model to Epic/stories description that I found interesting. It fills my gap with a systematic approach using a tool I love, mind maps.

It also helps deprecating a common challenge with use cases. When you get dirty with them, it is easy to write many that are not key for the main goal of the tool. Once you have many of them, it might become tough to relate and prioritize them.

Gojko propose to get rid of use cases and go directly from impact mapping to describing the epics/user stories. I have not been able to do so. It might be that I am simply too hooked to use cases and it is only a matter of time before I follow exactly the new process. It can be also that impact mapping is rigid compared to use cases so you can only get rid of them in a second/third iteration of the impact mapping process...

In any case, for Bitacora I started with an impact mapping. Once it was mature enough, I started from scratch doing the use cases. My impact mapping ended up containing 75% percent of the use cases.... and already structured. I assume that, in a collaborative design process, this percentage will be greater. I assume also I can get better with practice. So I am confident to adopt Impact Mapping exclusively. I might loose some use cases but I gain structure. Relevant but at this point ignored use cases will probably pop up later in the design process or during development. Mistakes in the prioritization due to structure deficiencies has a bigger (negative) impact along the product life cycle.


What is an Impact Mapping?



Mr. Adzic describes in his book Impact Mapping as "a mind-map grown during a discussion facilitated by answering the following four questions: Why?, Who?, How?, What?"
  1. Why: centre of the map, represent the goal of our application. It should include the desired achievemnt based on a key metric.
  2. Who: first level of the mind map, it refers to actors who can influence the outcome. These are the personas. At this point of the process, it is not mandatory to describe them in detail
  3. How: second level of the mind map, it refers to impact we are trying to create on the actors, that is, how the product change their behaviours.
  4. What: third level of the mind map, it refers to the scope, that is, what can we do to achieve the desired impact. Depending on the dimension and complexity of your product/service, this third level will be close or far away from a user story. You can either extend the What section into sub-levels (product approach) or define epics (scrum of scrums). 
You can get further information and diagrams in the website impactmapping.org I am still unsure about the direct relation between the What section and what engineers understand as a derivable. I prefer to reserve derivable as concept to a later stage (user stories) so they are analysed in detail by the product team, together with acceptance criteria. I find this arguable though.

Bitacora Impact mapping

Once you:
you should be able to understand the coming Bitacora Impact Map.

If not, I have probably done something wrong, which wouldn't surprise me :-) or you are not familiar with the problem to solve. Some of the ideas like "tag page" will be described in the user stories, so don't panic, we will get there.

Since these graphs has gone through several review processes, it might happen that the partial pics are not 100% up to date. The general impact map should be though.

Center and Level 1: Why? and Who?


The center lacks the quantitative goal. Since this is an ideal exercise, I will ignore it for now. We will go down to metrics and measuring impact at the very end.

At the end of the previous article, about personas, I wrote that for a introductory analysis, we could reduce them to 5. Since I went further, I considered groups and described personas for each group since we will use them at some point.

Level 2: How?

This is the second level that defines the impact for the product team members.


 This is the second level that defines the impact for the other personas.




 As you can see, the impact is described in the following areas:
  • Participation: the tool should promote an increase of interactions based on what each persona did.
  • Team/project follow up: bitacora should make following up a team or project activity easier.
  • Analysis: analize what the team is doing or has done should be easier with Bitacora. Self analysis is as important.
  • Report: Bitacora is designed to improve reporting in bothe, vertical and horizontal direction (matrix).
  • For Team Member, Bitacora should work as the central point for everyday (short term)  information.

Level 3: What?

For a easier visualization I have divided this level in three different images. The first one correspond to Product Team:



This is the third level for the Scrum Master persona:




 And this is the third image, corresponding to the other three groups:


Complete Impact Map

Here you have the complete map.

You might be wondering what are the icons for.

I mentioned earlier that one of the advantages of using Impact mappings is the structure. As a consequence, you can start to assign priorities to the different scopes which is very useful before writing down the User stories. 

Note:

The result of this process should in general be features, but in certain circumstances, at this point it could also be processes. I used the "help" icon to reflect them. his concept is arguable. I am at this point unsure if it will survive the story description.

Series of articles

  1. The diary (aka bitácora): towards alignment in distributed environment.
  2. Bitacora: environment definition.
  3. Bitacora: personas
  4. Bitacora: Impact mapping
The following article will describe the use cases I worked with so you can compare both approaches.

Sunday, May 17, 2015

Bitacora: personas

Introduction

This is the third of a series of articles about Bitacora. Please read the previous ones to get full context:
  1. The diary (aka bitácora): towards alignment in distributed environment.
  2. Bitacora: environment definition.

The environment description is incomplete without the personas. I like this concept for many reasons but the main one is because I get extremely annoyed by the use of the word user.

When designing a product/service, it is hard to agree on what a user is, specially in engineering environments. In marketing/product environments, in general is in the DNA of those involved in product design to differentiate buyers from consumers (users). Market segmentation is understood by default. In my experience, this is not always the case in engineering environments. 

I must admit that I do not use this concept every time since again, in my experience, engineers tend to show a high resistance to this concept, specially those who have direct contact to users, that is, those who work in open and collaborative environments. They tend to consider it mainly a simplification not of a representation.

Groups

We will assume there are 4 groups/roles involved in our scenario. Each group will be represented/formed by one or several personas. To label the groups, I will use names associated with roles you probably are familiar with, which is not a good practice since it limits their potential translation to a different scenarios. I do it so it becomes easier for you to identify them. The groups, based on its interaction with Bitacora are:
  1. Product team: professionals included in the team that will develop the product.
  2. Customer representative: professionals in charge of the requirements/backlog. Responsible for the product.
  3. Senior manager: professional in charge of the above groups.
  4. Supporting actors: people that interact with the team and are needed to develop/launch the product. They can be part of the company, the open source community that develops the technology the product is based upon or the sponsor.
If this blog series would try to reflect the Bitacora design process sequentially, the number of personas, even groups, should be lower, increasing them when necessary during the design or development process.

Product team


 I have considered 4 personas:
  •  Junior developer (jdev)
  • Senior Developer (sdev)
  • Artist (art)
  • Scrum master (smast)
persona_kareem
persona_james
persona_lisa
persona_byron
Artist represent non-technical profiles technical teams might include like domain experts, communication experts, etc. At this stage, defining 4 personas is not the right thing to do. Two (scrum master and the rest) would have been enough. only when going into details, the four personas will make sense.

Customer representative


I have considered two personas:
  • Product owner (pdm)
  • Sponsor (spon)

persona_earvin

persona_ac

Like in the previous case, one persona is enough for most of the bitacora design process.

Senior manager



I have considered a single persona, the engineering Vice President, which is the most common non-executive senior manager (EVP).
persona_michael

 

Contributor


I have considered three personas:
  • Marketeer (mark)
  • FLOSS community member (cmem)
  • HR representative (hrep)
persona_candece
persona_kurt
persona_tina

The Marketeer could also be any other professional within the company that interacts with the team. The FLOSS community member could also be an external consultant/domain expert contracted to help the team.


As in the previous cases, for most initial stages of the process we will follow, the first two personas could be considered as one. The third one has a very specific and limited role in this design, based on my experience, so it could be ignored or somehow included in the senior manager persona. I will refer to it in very few (but relevant) cases.

For a close follow up you might want to download the slides.

The cliparts has been taken from clker.com and 1001freedownloads.com If

Conclusion

As you can see, I have considered 11 personas, which is a lot at this point. But I started with 5 which must be considered the main ones:
  1. Engineer
  2. Scrum master
  3. Product owner
  4. EVP
  5. FLOSS community member.
So from now own I will refer to them by their names or roles. Based on your input and the process design development I might refine them.

Series of articles

  1. The diary (aka bitácora): towards alignment in distributed environment.
  2. Bitacora: environment definition.
  3. Bitacora: personas
  4. Bitacora: Impact mapping
The following one will describe the impact mapping. This section will be updated with the coming articles.

Saturday, May 16, 2015

Bitacora: environment definition

Introduction

This is the second of a series of articles about Bitacora. Please read the first one to get full context.

In order to understand the perfect implementation of a Bitacora we need to go down the road of a product definition. One of the first steps is to define the environment.

Bitacora is specially designed, although not exclusively, to distributed teams. So we need to first understand what do we mean by a distributed team.

Technically speaking, we need to define the environment in which our product will live. At this point, we will take it as a hypotheses, which should be the result of studying and defining our business model. But that is out of my scope so I will use as starting point an abstraction of the different environments in which I have worked. In most of them I have worked with imperfect/limited/working versions of (the ideal) Bitacora. 

Description of the environment

We will assume the Bitacora is designed to be used in a professional environment, a company that has four offices:
  • The headquarters are in Silicon Valley (UTC-8).
  • The main engineering office is in New York, US (UTC-5).
  • QA and services dept. are based in Hyderabad, India (UTC+5:30).
  • The fastest growing office is the London, UK (UTC).
Three years ago, the company changed its growth strategy due to the difficulties they were facing to attract talent. The increasing relation with some Open Source communities acted as catalyst for this change. So now, instead of relocating people, they hire the talent wherever is available, becoming what I refer to as a distributed environment.

50% of the people hired (not including sales and customer support) the last three years are home based. They expect to increase this number up to 75% in the coming three years.

The company was originally conceived as an agile company. Every product/service has been developed following Kanban or Scrum. One of the challenges they are currently facing is how to adapt those methodologies to:
  • The increasing relation with open collaborative environments.
  • The distributed nature of the teams.
At the same time, the company need to face an increase of the services associated to the products they are developing. This is creating a friction between processes associated to product development and services.

Due to the increasing interaction with open source communities, it is expected that this challenge will get more complicate due to the R&D nature of the work done in these communities, that incorporates new and different processes that the company do not has control upon.

But above all, the main challenge brought by the new distributed nature detected by senior managers is the the reduction of alignment within the company which has two clear symptoms:
  1. Appearance of silos.
  2. Complains by senior engineers and first level managers of lack of big picture and reduced horizontal communication among teams.
A new team has been formed within the company to develop a new product, which is an evolution of a former one, based on a technology developed by an open source community. The new product is sponsored by a reputed German company that is building a complete solution for a vehicle industry leader.

This team is multidisciplinary and, following the nature of the company, heavily distributed.  It will have direct relation with the open source community the product is based on, the sponsor and several key stakeholders within the company.

Finally, since this is product considered key for the future of the company, senior management want to evaluate the work done on a regular basis.

You, reader, and I, are in charge of, using this team as test case, designing a tool that improve alignment keeping high levels of autonomy, using agile principles while reducing the impact on existing company processes to the minimum.

That tool is Bitacora.

Why this environment?

I see the above environment as the maximum common denominator of the key ones I have worked in, that is:
  • Team distributed in several offices in the same time zone
  • Team centralized in one office with a significant % of remote (home) work with a remote product manager, all in the same time zone.
  • Team distributed in two offices and several remote team members in different time zones. 
  • Team completely distributed (no office) on two correlative time zones.
  • Team completely distributed across the globe in several time zones.
The possible combinations increase if we introduce culture/origin/native language/profile of the team members or sponsors, the nature of the company/product/service being developed, my role in the team and the relation of the product with collaborative environments. 

I also wanted to choose an environment that might sound familiar to my blog readers. This scenario is probably not the one where Bitacora would have the highest impact, but again, my main goal is explaining the ideal solution, not creating the best business plan.

Series of articles

  1. The diary (aka bitácora): towards alignment in distributed environment.
  2. Bitacora: environment definition.
  3. Bitacora: personas
  4. Bitacora: Impact mapping
The following one will define the personas. This section will be updated with the coming articles.

Wednesday, April 29, 2015

The diary (aka bitácora): towards alignment in distributed environments

Introduction

There is a virtuous circle that many think is key for any organization to achieve success:





I want to focus on autonomy and its relation with alignment.

I do not know any single smart person that do not like autonomy. Definitely I want it and most of the managers and developers I have worked with too. There is no doubt that we achieve the best results for ourselves and the organization we are involved with with great doses of autonomy.

Autonomy though, as usual, requires high levels of coordination in order to be sustainable over time. It requires mechanisms of check and balance against a plan. Take a look at Scrum, for instance. It proposes to do checks and balances against the backlog every 2 to 6 weeks aprox., at the beginning/end of every sprint And it forces also to stablish a daily check and balance against the micro plan every day. That is a lot but it is the price to pay for getting autonomy.

And this is because the success of a group completely depends on the alignment of its members. Our tendency of "taking our own way" (entropy) is so high that organizations need people and many processes just to make sure everybody fight hard against it.

Definition and a little bit of history

I think it was 2004 when I started my relation with Fotón S.I. crew. Fotón is an open source company from Gran Canaria, founded in 1998, lead by Mike Vazquez and, back then, also by Gonzalo Aller.

Working with Fotón SI had a huge impact on the way I perceive management. It worked as a catalyst. I learned with them in a few months what takes ages for others.

Fotón was formed by a group of very talented and young hackers. The decision processes were very participative and transparent. Let me put you one single example: the basic accounting of the company was open to every employee, including salaries.

In their peak times they were around 20 people but it was strange to see more than 5/6 in the office. Most fotonians worked from home most of the time. They were based in Gran Canaria and my little company was in Tenerife. It was a distributed environment.

Going back to Foton, a key variable that balanced autonomy and alignment  in that distributed environment was THE DIARY, a simple concept but very effective.

Most people right now are probably thinking about that little notebook that needed a key to be opened where some boys or girls write their thoughts at early ages. Obviously that idea has little to do with what I am talking about. The term bitácora, a short way in Spanish to refer to the ship's log, is closer to what I am talking about.

A bitácora is a log chronologically organized, written by the ship's captain, kept in a wel define place where the most relevant events related with the ship, the crew and the journey where described. It had two main goals:
  • Analysis.
  • Report.

By writing on regular basis what happened in the ship, captains were creating a tool that allowed them to learn from past experiences. It was a very precious treasure, so preious that needed to be destroyed in case the ship got into the wrong hands. It was way more than the Black box of a plane. It captures the most important knowledge. If the captain got sick, for instance, or died in combat, his replacement used the bitácora to analyse the past history. Many of you are familiar with this concept since it is popular among scientists, archaeologists, etc.

The bitácora worked also as a reporting tool. After a few months sailing, it was impossible for any captain or officer to remember details about the trip if they were not written in real time. Providing a report about the trip was impossible without the bitácora. One interesting point was that, once written, no entry could be modified. If the captain made a mistake, he needed to create a new entry correcting the mistake, like in a check-book.

Fotón was a pret-a-porter development company organized per project. Each project had its own diary/bitácora. Following its culture, everybody had +xrw permissions in every bitácora. Agreed labels were used for different purposes: identify each user, assign relevance a specific entry through colours, etc. This idea was implemented in a hacked Twiki, integrated with Request Tracker and a mail service for notifications. A very smart, geeky, simple and efficient implementation for the time. Like all the wikies back then, UX was not the main feature but for console lovers.....that was no issue.

So Fotón extended the concept of the bitácora to a new level, going from an individual oriented tool to a group one, that is, they took a collaborative approach with a great result.

I still remember the day they explained and showed it to me. I was amazed, excited and cautious about its scalability. I was already used to write regularly what I was doing so the main hurdle did not applied to me: I had the habit. The collaborative approach was new to me though, together with the implementation.

How come something so simple has such a high impact? What makes reading what others do, think or feel so valuable? What is the relation of writing what you do and being efficient as a team? What a diary has to do with alignment? And here is the question that most people that face this tool/process for the first time ask: isn't it the diary a tool to control me?

Let me start with the last one. The bitácora is a tool to control yourself. If you think you do not need to regularly check and balance your actions against your plans, your colleagues expectations and your company goals is simply because you are not senior enough, period. The diary is just a way to achieve that, like the stand up meetings or the burn down chart.

You decide what kind of information you should include in the bitácora. Here is the main rule. Knowing that is open to those you work with, add what you think is :
  • Relevant to you.
  • Relevant to your peers.
  • Relevant to those you interact with like, managers, other teams, customers, etc.

Do not add information that only one person should know about or that you would not say in a team meeting. Simply use your common sense. For instance, if I am frustrated about something.... I add it in the diary if I want to share it.

Fotón used to open the diary about a specific project to customers and third parties involved on it and it worked beautifully. Of course once in a while somebody wrote something inappropriate, but the benefits of these diaries were so high that assuming the damage was out of question. It became an outstanding engagement tool, as good as the best sprint planning meeting.

Very soon I experienced myself the benefits of getting the habit of writing regularly in those project diaries I was involved with, reading what others wrote and interacting with them through the diary. I got a sense of control of my day to day work and, even better, of what everybody else was doing. Control in a good way. Sync meetings were reduced, asking for weekly reports made no sense,... we were a more efficient, coordinated... better aligned organization, even in our distributed environment, despite working in several projects at the same time, with different customers, third party companies or different working schedule.

Conclusion

In many ways Fotón diaries laid upon two ideas that today are very popular: semantic and social (in a twitter way). I believe that the diary is a key complement to agile methodologies, specially in distributed environments.

In my next article I will describe in detail the concept and what the perfect implementation should look like, based on my experience since I have used it widely since then.

Series of articles

  1. The diary (aka bitácora): towards alignment in distributed environment.
  2. Bitacora: environment definition.
  3. Bitacora: personas
  4. Bitacora: Impact mapping
The following one will define the personas. This section will be updated with the coming articles.

Sunday, April 12, 2015

Apply agile methodologies to upstream development environments.... if you can.


 Introduction


When the Agile Manifesto became popular and based on them, agile methodologies like Scrum, XP or Kanban, upstream development was in its early stages as collaboration ecosystems of companies.

Only a few for profit organizations embraced developing upstream back then. Most of them were small and heavily influenced by FLOSS engineers vision. Free software communities were basically driven on personal basis or the very lucky ones, together with "sponsored developers". In general, these ecosystem were not part of companies strategies.

Today, more and more companies are getting fully involved in community projects as stakeholders, not just consumers or simple contributors.

They frequently start as consumers, then, little by little they become "upstreamers", that is, they share/publish their code with the goal to have it merged (upstream code). Not without effort, many of them become successful contributors.  After some time, some of them end up understanding that is "cheaper" to play by the project rules. In summary, they learn to become good citizens.

A subgroup of the above companies end up including these collaboration ecosystems as part of their own strategies, going from contributors to  key stakeholders. A necessary step to achieve this goal is to work upstream.

Walking this path present many challenges. One of the toughest ones is related with the differences in development methodologies used internally (mostly agile) and those used in the collaboration ecosystems.

There are two fundamental variables that, in my opinion, determine this challenge:
  1. Environment
  2. Culture

Challenges


1.- Environment


There are two dependent variables that were not taken into account (or just partially) when the agile methodologies were defined, that are relevant in upstream development:
  1. Community projects are global environments, that is, contributors are located in different "offices", frequently in different time zones.
  2. Probably due to the original amateur condition of early contributors, together with the "distributed condition", the development processes (so the tools) in most mature community projects, consider, manage and tolerate high levels  of latency.  "Real time" is restricted to IRC discussions and events/conferences.

These two factors has made open source what it is today. They have been "success factors".

Agile methodologies do not embrace "distribution" environments. The widely accepted recommendation is that teams should share a physical space. It is way more than a recommendation. It is somehow a requirement.

The second case, "latency", is considered by agile methodologies as a waste. It is not tolerated.

2.- Culture


Free Software was born as a reaction to a system that promoted corporation interests over developers, so users. The agile movement was a reaction to those methodologies that put process first, not people. Hence, it is obvious that both movements share a lot: people first

This is reflected by some when saying that FLOSS development is agile.

In my opinion, there is a big difference between what agile methodologies and what Open Source development propose in terms of principles.

Agile methodologies promotes a strong team culture. Open Source was born "based on champions". FLOSS culture normally applies the meritocracy concept to individuals.  Open Source projects are organized around contributors, around specialists, not around teams, as we understand them in corporate environments.

This is no surprise since Agile was born in companies/corporations and Open Source was born as a viral movement, grown "by aggregation".

The conflict


In my opinion, the more the industry embrace open source, and as result, open collaboration, the higher the conflict developers and managers will face due to the above challenges.  Companies are becoming more distributed environments and are working more and more upstream, instead of simply being consumers or occasional contributors.

In consequence, it would not surprise me if we hear more and more about  "corporate development methodologies" (a.k.a. agile) vs. "upstream development methodologies" (a.k.a. FLOSS).

Scrum, XP, Kanban -ish fans will need to face those challenges and find solutions in order to succeed in open collaboration environments. In the same way, based on the increasing influence that companies are gaining in these ecosystems, FLOSS methodologies in a few years will differ from what we knew 10 years ago.


This conflict will not be (is) about a R&D vs a product/service vision, it is not about creativity vs efficiency, it is not about micromanagement vs autonomy or teams of juniors vs specialists either. It is about methodologies applied to specific environments and its limitations. Maybe a simple update of the most successful agile methodologies will do the job.... or maybe we need to revisit some of the principles.

If you got here, maybe you want to take an extra step and answer these questions. I would appreciate it:
  1. Do you perceive this conflict as I do?
  2. Am I missing other key elements in the diagnosis?
  3. How do you think we can adapt agile methodologies so they can be adapted to FLOSS environments?
  4. I am interested in knowing how you adapt agile methodologies to overcome the above challenges. I plan to write about my experience these coming days.

Wednesday, February 18, 2015

Where the corporate and the upstream worlds meet.... or collide.


From the corporate world I frequently hear how hard it is to predict and track what upstream developers do. On the other side, developers that work part or full time  upstream frequently underestimate the need for communicating what they do in a way that enable others (or themselves) to provide deadlines and effort estimations. Upstream and product "time lines" and cultures often differ too much to be compatible under the same environment.

Developers that come from the "product" side of the story often refer to agile methodologies like Scrum as a good approach to solve this collision. It is unclear though the applicability of this and similar methodologies to highly distributed environments, where latency is high. Although most accept today that FLOSS development is agile, I find hard to assume that we are close to find good answers when it comes to development methodologies and upstream.

There are several different scenarios I have worked on where I have to question everything I knew up until then about this particular topic. In a couple of occasions, with a bigger or smaller community, the people I directly worked with were upstream. This allowed us to define up to a great extend the methodology we worked with.

In Linaro now this is the case for a particular project. But in general, in my Group we work upstream (in the Linux Kernel), that is, we cannot define the methodology since it comes defined by the project itself. It is not the first time I face this situation although I never did at this scale.

When you are upstream, the methodology used by the core team, together with how the project work packages and workloads are organised heavily influences the success in getting contributions from third parties. Together with other key variables, how you manage latency determines how many people can follow you and potentially contribute. In summary, the faster you move the harder it becomes for contributors to collaborate with you.

Hence agile methodologies come with a high risk: isolation. And this is obvious by just analyzing the prerequisites associated with the methodology itself.

Obviously this does not mean that scrum and other methodologies are not great ones. I am just pointing challenges associated with apply them when working in a community that you drive. You need to balance the efficiency of your team with the contributions you might potentially loose from externals.

The challenge is even bigger when you are just a participant in a community. In some way, your team goes beyond your colleagues at work. Latency is frequently too high to even consider yourself and the people that works close to you "a team" in the classical sense of the word.

If your own team is distributed, even if they are in the same time zone, then we are talking about the real deal, the master challenge for managers and developers. 

Imagine now that your team is formed by 10 people distributed in 6 different time zones in 3 different continents. Then you take one of those cool books that tries to explain how to create great software using this or that methodology and, in order for you to finish it, the writer must be a really good one or you really need to be an avid reader :-) 

Still you need to make plans, to predict when a feature will be finished, when it will be merged, you still need to manage dependencies, expectations, budgets.... you still are tight to somebody else needs and requirements. You still need to fulfil expectations despite the fact your control over the environment is limited. You have team mates that depend on your work, that needs to know what you are doing and how.....  you are still part of a business no matter if you develop upstream or not.

How can you make compatible the upstream development processes used in the community you are contributing to with the way your company works? And how about your customers and partners, if you have direct relation with them? How can you take the good things that agile development propose and apply them to an environment with high latency, where one of your bigger challenges is to have an efficient team meeting, since people are distributed across the planet?

At Linaro, the engineering management team, together with our PMO, are taking a close look at this challenge, in order to iterate from our current system to a more efficient one, better adapted to our new reality, after our significant growth during the past 18 months. The idea is to find a place where corporate and community processes meet... without colliding. You cannot stop adapting, innovating, trying new things..... or you pay a high price.

One of the great things about Linaro is that we are a unique environment so we can innovate at various levels, not just at the technology one.

Open Sorce Forum at CebIT


For those of you attending to CebIT, let me tell you that I am giving a talk about what we do at Linaro on Monday March 16th at the Open Source Forum. It is the first time I talk about my current job in public so I am very excited about it. It will be also my first time in CebIT so double doses of excitement. And yes, I will talk a little bit about Linaro new initiative, 96boards.

Core Dump


The past months we have published a few articles about what we do at Core Development Group at Linaro. Our blog is called Core Dump. Check it out.

Friday, August 15, 2014

Team blog: more than just a blog.

Motivations


Creating high quality contents takes time. A lot of people write nowadays but very few are writers. In the software industry, most of those who write very well are in the marketing side not on the technical side.

The impact of high quality contents is very high over time. Engineers and other profiles related with technology tend to underestimate this fact. When approaching the creation of contents, their first reaction is to think about the effort that takes, not the impact. Marketers have exactly the opposite view. They tend to focus in the short term impact.

Successful organizations have something in common. They focus a lot of effort and energy in reporting efficiently across the entire organization, not just vertically but horizontally, not just internally but also externally. Knowing what others around you are doing, their goals, motivations and progress is as important as communicating results

One of the sentences that I do not stop repeating is that a good team gets further than a bunch of rock stars. I think that a collective approach to content creation provides better results in general, in the mid term, than individual ones. Specially if we consider that Free Software is mainstream nowadays. There are so many people doing incredible things out there, it is becoming so hard to get attention....

Technology is everywhere. Everybody is interested on it. We all understand that it has a significant impact in our lives and it will have even more in the future. That doesn't mean everybody understands it. For many of us, that work in the software industry, speaking an understandable language for wider audiences do not comes naturally or simply by practising. It requires learning/training.

Very often is not enough to create something outstanding once in a while to be widely recognized. The dark work counts as much as the one that shines. The hows and whys are relevant. Reputation is not in direct relation with popularity and short term successes. Being recognized for your work is an everyday task, a mid term achievement. The good thing about reputation is that once you achieve it,  the impact of your following actions multiplies.

We need to remember that code is meant to die, to disappear, to be replaced by better code, faster code, simpler code. A lot of the work we do ends nowhere. Both facts, that are not restricted to software, doesn't mean that creating that code or project was not worth it. Creating good content, helps increasing the life time of our work, specially if we do not restrict them to results.

All the above are some of the motivations that drives me to promote the creation of a team blog wherever I work. Sometimes I succeed and sometimes not, obviously.

What is a team blog for me? 

  • It is a team effort. Each post should be led by a person, an author, but created by the team.
  • It focuses on what the team/group do, not on what they will do, what they think or what they want. Why? because a team blog is way more than a promo tool, it is also a reporting one.
  • It is supported by a communication expert as editor, not just to increase the quality and potential impact of every content, but to increase the team skills as writers in a practical way.
  • It is management driven because it requires to embed it in the team processes and group dynamics. So it requires analysis, a clear goal, follow up and activation energy, specially during those moments in which the team workload is high, so writing is not perceived as a high priority. 
  • In order to ease the identification of the team with the blog and vice-versa, the promo aspects should serve the team and not the other way around. This is a key difference between this action and other common marketing efforts done by every organization.
  • Selecting the right topics to write about is not a question since, as mentioned before, the team blog is about what the team is working on. The question is about when to publish, together with the approach used to create the articles. Again, a management point of view is essential here. That view can come from different areas of the organization or within the team, but has to be there to take full advantage of the effort.
  • It feeds engineering reports. It saves time on this task in the long run.
  • If the goal of the action and the vision brought up by the editor are set up correctly, the team blog should also feed the technical marketing and customer engagement efforts of the organization/business unit. 
  • In the mid term, the blog serves as input for evaluating the performance/accomplishments of the team as a group, when associated with the key objectives previously defined. This is true not just from a management or customers perspective. What is more important, the team blog serves as evaluation input for the team itself too. The retrospective that this action can provide is essential for the growth of any team. 

Core Dump

I am pushing such action once more, this time at Linaro. The name of the blog is Core Dump. My presentation describes the motivations and goals that drives it. There is a talented group of professionals behind it so I hope that in just a few months we will be able to evaluate the results. I am confident we will succeed since we have every required ingredient.

Sunday, August 10, 2014

First weeks al Linaro and other things

I few weeks ago I announced I was joining Linaro. I work there as Director of Core Development Group. I moved from Prague to Cambridge (the original), that is, from continental to oceanic climate. From dry, cold in winter and hot in summer to wet, soft in summer and above zero most of the winter. In theory an improvement, you might think. Well, depending on much it rains. I will tell you better in spring.

A few days ago The Mukt published and interview where I explained a little what is Linaro and what do I do as Core Development Director. 

Core Development Group


I can add that Linaro is an engineering focused organization, divided in Engineering Groups. Some of them, like the one I am part of, are formed by several engineering teams, some of them called Working Groups. Core Development is formed by four:

 You can find more details in the Core Development wiki page, at Linaro Wiki.

These first few weeks I have gone through the natural landing process, meeting my colleagues and managers, knowing how we operate, learning about my responsibilities, the work engineers are doing, the plans for the future, analysing our internal processes, etc. Nothing unusual in these cases.

In July I had the opportunity to attend to the Linaro Kernel and Power Management Sprint, hosted by one of our Members, ST, in Le Mans. It was a very interesting week.

Linaro Connect


My following event will be Linaro Connect, in San Francisco, USA, in September. Linaro Connect are the events where all the Linaro employees meet. Those of you who are familiar with the Ubuntu Developer Summit knows what I am talking about. Linaro Connect takes place twice a year in a different continent and it is also an opportunity to have a direct contact with our Members.

Factory as a rolling release: openSUSE development version

A few days ago it was announced that Factory moved to a rolling release model. So the first step of the 2014/2016 plan has been completed. I was very happy to see that the openSUSE team could lead the execution of this relevant step for the distro in time. The Development version of openSUSE is now a reality that not just can increase the overall number of contributors, but also bring significant innovation to SUSE Linux Enterprise integration process. Congratulations. I am very proud of being part of the team. I will always be.

I would like to specially congratulate Roland Haidl, the Director of Communities at SUSE. The most important (and hardest) thing you can get from a manager is trust, and the openSUSE team had it from him to build a good team, support the changes the team went through back in 2012 (tough times), stand strong behind the new strategy defined in 2013 and support the team during the design and execution of this first milestone. And he did this without making noise, letting the results speak. A management handbook success.

Personal challenges


On the personal side, relocating has taken most of my energy the last two and a half month. But I have managed to do/plan other things.

LinuxCon Europe

I am part of the LinuxCon Europe Content Committee. The last few days I have been evaluating, together with my colleagues in the committee, the abstracts presented. I has been an interesting work

LinuxCon Europe have very promising keynote speakers but the whole program will be filled up with first class contents.

Sometimes you never know what lives will bring you. I was invited to be part of the committee before joining Linaro. My work now is very related to the Linux Kernel and suddenly, last week, Linaro becomes a Linux Foundation Member. I do not want to think too much about coincidences vs. destiny but....

Akademy 2014

This year Akademy has moved to September and the dates collide with a personal compromise (a wedding) and the preparations for the Linaro Connect so I won't be able to attend. It will be the first time I'll miss it since 2005, when I attended for the first time. I am very sad about it. I tried but...

And it is specially sad for me because the new Treasurer will have to present the financial report about the past 2013, when I was the Treasurer. Being there should be the right thing to do. But I simply cannot make it.

I already missed Akademy-es 2014, that took place in Málaga and was organized by my former colleague Antonio Larrosa and sponsored by my former company, SUSE. I am not running away from KDE, I promise. The Calendar is working against me, that's all.

TEDxLaLaguna

I am a TED video consumer since some years ago, not many. Suddenly, I received a mail from a colleague at college giving me the opportunity to start.....and at home, in the Canary Islands.

Of course I want to try!

So in October I will be in Tenerife giving a TED talk in the local event TEDxLalaguna. Obviously I am already preparing the talk. Let's see how it goes.

As you can see, it is going to be an intensive year, after all.