Wednesday, October 26, 2016

Automotive, what an opportunity for KDE!

During the last year I've focused a significant part of my effort on driving the GENIVI Development Platform, together with some Codethink colleagues and other GENIVI professionals and community members.

What is GENIVI Development Platform?


GENIVI Development Platform (GDP) is a project and an outcome.

As a project, it can be defined as the delivery side of the GENIVI Alliance. Today is a fairly standard Open Source project, done in the open following many of the most common practices any FLOSS developer would expect.

As an outcome, GDP is a Linux base distribution (Poky) derivative, built with Yocto, that integrates the software that GENIVI community (automotive professionals) develop as Open Source software.

It still a small project but the quality of the platform and the number of people involved has grown this last year significantly.

GENIVI Alliance is a consortium of +140 companies so obviously most of the overall effort is done by paid developers. Changhyeok Bae (community member) together with Tom Pollard and Robert Marshall (Jonathan Maw before him) from Codethink Ltd, constitute the maintainers team, who are responsible for the integration, testing and release of GDP.

These guys are supported by people like myself, doing coordination, marketing, documentation, IT services, infrastructure, testing and many other key tasks which provides the project a level of robustness and scalability that any serious attempt of this nature requires nowadays.

I am interested, where can I get more?


You can find more general information about the project in the following resources:

GDP-ivi9 is the current stable version although we have moved so fast this last year in terms of the software shipped, that I recommend you to try the following:
  • If you are interested in a solid base system, try GDP 11 RC2
  • If you are interested in checking the latest UI and demo applications, try GDP 11 RC3
  • If you are interested in building from scratch you own images with the latest software, do it directly to GENIVI's rolling release, called Master for now. You can get the latest software there. It should most ly work since we put stabilization effort on it, following the openSUSE Tumbleweed mindset in this regard.
Currently GDP supports RPi2 & 3, Minnowboard MAX and Turbot, Renesas Porter and Silk and Qualcomm Dragonboard 410c. GDP ships Qt 5.6 at the moment, since it is based in Yocto 2.1...

...which makes GDP a great target for KDE software, specially for Plasma.

GDP and KDE

Putting the effort on having KDE well supported in Yocto would provide the project a third life, landing on an industry that is heavily investing in Open Source with a key piece of software, with no clear competitor today in the open.

It would revamp the interest of many KDE developers in porting their apps to embedded/mobile environments and would bring attention to the project from Qt professionals all over the world. Currently KDE is significantly better than anything else that is open in automotive. It would just require the effort to include it and maintain it in Yocto, which is not small, and adapting Plasma a little initially, not much.

GENIVI launched a Challenge Grant Program that might help to put some funding in the equation ;-)

Whatever effort done to put Plasma on Yocto (so GDP) would also be picked up by GDP's competitor, AGL UCB (Auto Grade linux Unified Code Base), the Linux Foundation automotive group Linux (again, Yocto) based distribution. So there would be at least two players for the cost of one.

It wouldn't surprise me if Qt companies would jump in on this effort too. In order to play in the open filed today, they need to Open Source their products, which is a big risk for most of them. Playing with KDE, which is based in the technologies they are familiar with, would be simpler for them. I bet the Qt company would be heavily interested in promoting this effort. It would help to dissipate all the pushing back from the automotive industry to the current Qt license model, GPLv3 based. And it would do it in the best possible way, by providing great ready-to-use software with no competitor.

I have been one year preaching about how big the opportunity currently is for KDE, but this is not the kind of challenge that can be sustained on volunteer basis, sadly, since keeping KDE up to date in the Yocto project would require a high level of commitment from KDE as a whole. The community probably needs first a small success story and some company/corporate push before really jumping on it, I think. The support of a couple of KDE or Qt companies would catalyze the effort.

GENIVI and AGL make a significant promotion effort around the world within the automotive industry, participating in forums where KDE is unknown. Many companies that currently develop close source Qt applications for automotive would be interested in KDE which would increase our potential targets for our Corporate program.

Having KDE on GDP and AGL UCB would increase the incentive of developing new applications for many of our young developers who currently do not have automotive as a "professional target". 

Companies like LG, Renesas, Bosch, Hartman, Intel, Jaguar Land Rover, Toyota, Visteon, Fujitsu, Mitsubishi, Volvo, among many others.... are key stakeholders of GENIVI and AGL. Isn't it this attractive enough?

A success story like the one I am proposing would be yet another example of how KDE can play a key role for the Qt ecosystem. Sadly not everybody in this ecosystem understand what a great "tool" KDE can be for them. After a hit like this one, it would be undeniable.

Think about the exposure, think about where we are today... Something like this would place KDE where Unity, Android (AOSP) or GNOME are not... yet. I believe this is the kind of strategic decision that would change KDE future. But also the business perspective of those companies (specially Qt ones) who would get involved.

Let's do it now... or somebody else will.


Update(29/10/16): to find out about the state of KDE in Yocto please read the comment to this article from Samuel Stirtzel. Thanks Samuel.

Tuesday, July 05, 2016

Moving from a traditional product/release focused delivery model to a rolling model

The past few weeks the GDP delivery team together with some key contributors, has been working on a not very visible but still important change. The GDP project has put the basis to turn GDP release based delivery model to a "rolling" one. My colleagues will provide in a coming post the technical details behind this change. I want to provide a higher level view of what is happening and why.

Some background


GDP was born as a "demo" project. The main goal was to provide a platform to show the software components for automotive that the different GENIVI Expert Groups were developing. This was done through a delivery model focused on publishing a stable and easy to consume version of the project every few months, a major release.

Strictly speaking, GDP is a derivative. It is based on poky and uses Yocto tools to "create" the Linux based platform, adding the different components developed by the GENIVI Alliance together with upstream software. For the defined purpose, the release centric model works fine, especially if you concentrate your effort is very specific areas of the software stack with a small number of dependencies on the other areas, and a limited number of contributions and environments where the system should work.

During this 2016, the GDP has grown significantly. We have more software, more contributors, more components and more target boards to take care of. Although the above model has not been not challenged yet, it was just a matter of time.

As I explained in two previous posts [1][2], the GDP is moving from a being a Demo to a Development Platform. Changing the mission means changing the goals and the target group, which implies the need to adjust the deliverable to meet the new expectations.

So, right after the 14th AMM, the Delivery Team decided to change the delivery model to better meet the new mission, providing developers the newest possible software with the an increasing quality threshold. At the same time, in order to increase the number of contributors, the GDP needs to provide a new solid platform every once in a while. That should be done trough a solid release.

What is a rolling delivery model?


The key idea behind a modern delivery release model is to ensure that the transition from one stable release to the next one takes an affordable effort. I will put an example to picture the idea.

Problem statement


Imagine an organization that publishes one release per year. Let's assume that a particular release included 100 patches developed by employees and, during the lifetime of the release (1 year too), another 100 patches were added to the product as bug fixes and updates. At the end of the release lifetime, the product includes 200 patches that define the value the product provides to customers and users.

Either for technical or business reasons, a year later it is time to upgrade. Our organization has to create a new Linux based system with newer upstream code and they have to integrate the patches from the previous release plus the updates and bug fixes developed for the coming release.

After a simplification process done by engineers, the number of patches needed to be integrated in this newer base system is reduced to 150. The organization also wants to add to this new release another 100 patches that represent the new features they have been developing during the last year for this new version.

The delivery team now has to integrate 250 patches in the new base system, 150 of them coming from the previous release. One might think that the effort required to do this is 2.5 times the effort invested in the previous release. Maybe you think that the effort is not so high since some of the patches have been developed thinking about the new base system. There are many other considerations like this one that might affect the initial estimation. This example is obviously a simplification.

However any experienced release manager will tell you that moving patches integrated in an older system base onto a newer one (forward-porting) requires additional effort, beyond a linear relation with the number of patches. Forward-porting is the "road of hell". Iterate this example a few times and you will understand why there are so many organizations out there that have as many people focusing on delivery as they have in development. They migrated to Linux base system keeping the traditional delivery model they had while working with closed source software.


Release based delivery model

Possible solutions


One of the paths to improve the situation is upstreamming those changes that affect generic components. Some companies also upstream their new features early in their development process, generally looking for wider testing, or after they have been released to customers, to increase adoption and reduce future maintenance effort. This is definetly a must do.

From the delivery perspective, the most popular way to tackle the problem though is reducing the release cycle, so the number of patches to forward-port in each release is smaller. The development time and the maintenance cycles are also smaller. The same applies to the complexity of the forward-porting activities. "Jumping" from one release to the next one is easier to do. Add automation of repetitive tasks to this recipe and you feel you have a win.... for some time.

The journey through the "road to hell" becomes more comfortable, but our organization is still getting burned, even in the case that our customers and ourselves can digest releasing frequently. We all know how expensive and stressful a release might become.

The most suitable option to achieve sustainability while scaling up the amount of software an organization can manage without releasing more often than your market can digest is to change your delivery model.

Rolling delivery models are a serious attempt to solve this problem, putting integration as the central element instead of the software itself.

This model is not new. Gentoo has been doing it forever, but it was Arch Linux who implemented it in a way that immediately attracted the attention of thousands of developers. Still it was a model with no hope beyond hardcore Linux developers. openSUSE brought this model to a new level by implementing a process which output was stable enough for a much wider audience, and compatible with the release of a more stable and a commercial releases. Nowadays there are other interesting examples out there that commercial organizations can learn from.

What is a rolling model?


It is still hard to define but essentially it is a process in which ideally you have one continuous integration pipeline as the one an only entry point for the software you plan to ship. Releases then become snapshots of all or part of the software already integrated after going through a specific stabilization, deployment and release process.

So ideally, if you release a portfolio, you integrate only once, reducing significantly the costs of having different engineers working on different versions of the same software and forward-porting, among other benefits.

Rolling delivery model


So a rolling delivery model is a lot more than a continuous integration chain, although that is the key point.

Please have in mind that this is an oversimplification. This description doesn't go into detail on other key aspects like maintenance cycles, how upstreamming affects the process, strategies towards updating the released products, etc.

A transformation process that takes an organization from a release centric model to a rolling one is about doing less and doing it faster, so less people can handle more software with less pain, allowing more people to concentrate in creating value, developing new and better software instead of just shipping it.

Back to GENIVI


Moving from a release centric to a rolling model is hard work. Frequently it is easier to start all over again. Since the GDP is still a relatively small project, we can afford going through the transformation process step by step.

The first stage has been creating that single integration chain and treating GDP-ivi9, our latest release, and those that follow it, as a deliverable of what we call today Master. Ideally, no single patch will be added directly to the release branches. They should come from Master. That way, we reduce (ideally to zero) the effort of forward-porting of patches while putting in the hands of our contributors the latest software on a regular basis.

To do so, we are in the process of adapting our simple processes and CI system to the new model, GDP repository structure, the wiki contents, the task management structures, several key policies, our communication around the project...

The GDP will face a very interesting challenge since this model needs to be proven successful for a derivative. If we are able to move fast enough, it will come the time in which we will need to decide if GDP keeps being a derivative or it becomes upstream, that is, either GDP limits the delivery speed based on the Poky release cycle, or we work upstream with the Yocto project to increase our delivery speed.

That is a good problem to have, isn't it?

If (almost) everything goes right, after adding a few needed services in GENIVI's infrastructure and ensuring the updated software is in compliance with selected verification criteria, the same number of people will be able to manage and deliver more software. And once the new processes become more stable, automation will not just increase efficiency, it will boost the project by allowing GENIVI to achieve goals that only big organizations with large delivery teams can do. This is the kind of transformation that takes time to consolidate, but has a huge impact.

Based on my experience, I believe that if GENIVI is able to sustain this effort and keep a clear direction the next couple of years, the benefits of moving towards a rolling model will be noticeable even outside the industry.

This blog post was originally published in the GENIVI blog site on 2016/07/04. I have adapted the formatting to adapt it to Blogger. The content should be the same.

Monday, May 16, 2016

Embrace Open Source culture: the 5 common transformations.

Article originally published at Linkedin on May 15th 2016.

This is a story of what I have lived or witnessed a few times so far. A story of an organization that used to consume, develop and ship proprietary software for many years. At some point in time, management took the decision of using Open Source. Like in most cases, the decision was forced by its customers, providers, competitors... and by numbers.

A painful but unavoidable transformation was required.

 

1.- Open Source consumer


Engineers had to learn a new system, adapt or re-write those features that used to made the organization unique, together with many other painful actions. It was expensive at the beginning but, due to the cost reduction in licenses and the change that Linux represented in the relation with providers, in a few years it was clearly worth it. And if it wasn't, it didn't matter since it was what the market demanded. There was no way back

Every software organization has gone through their unique journey, but the final sentence of the story has been the same for all of them: they became Open Source consumers.

 

2.- Open Source producer


This organization gained control over its production and, by consuming Open Source, it could focus many resources in differentiation, without changing the structure, development and delivery processes. At some point, it was shipping products that involved a significant percentage of generic software taken “from internet”.


It became an Open Source producer.


You can recognise such organizations because they frequently create a specific group, usually linked to R&D, in change of bringing all the innovation that is happening "in the Open Source community" into the organization.


Little by little this organisation realised that giving fast and satisfactory answers to their customer demands became more and more expensive. They got stuck in what rapidly became an old kernel or tool chain version.... Bringing innovation from “the community” required back-porting, solving complex integration issues, incompatibilities with what your provider brings, what your customer wants.


So they have to upgrade.


This organization will be able now to take advantage of all the common features and compatibility that the new kernel, the new tool chain... brings. But, guess what, forward porting all the differentiation features this organization has developed, all the bug fixes, is so much work and so complicated that the challenge put the organization at risk.

 

 

3.- Open Source contributors


The organization feels now the downsides of becoming a blind Open Source consumer and producer. Execs feels like when a bubble explodes and they are inside of it. They has less control than they thought, which turns out to be expensive, and what is worse, they lack the expertise within the organization to gain it....

But, after struggling for some time, this organization survived, which means that it has learnt some lessons:
  • Upstream those features that are not differentiation factors any more.
  • Increase the investment in that reduced groups of rock-stars that are up to date of what's going on "in the different communities".
  • Invest in those Open Source projects that develop the key software you consume.
  • Even better, start your own Open Source project to promote your technologies and be perceived as a leader...
  • Reduce the upgrade cycle, so the "upgrade pain" is lower. As a side effect, the organization has the opportunity to increase the cash flow when doing two smaller upgrades instead of a big one. At the very end, the real profit comes with the first update, not with the new version, right?

This organization ended up upstreaming features when they could, normally very late, because “they do not have time”, frequently assigning that task to young inexperienced developers or, even "better", subcontracting it, which is "cheaper".

You can recognise that this organization has gone through the described process when attending to conferences given by any of its executives. They cannot stop talking about how much they contribute to this and that community, about how awesome the community is, how important it is to be open and share.... they are referring all the time to communities/upstream as "us” and “them". They think they got it, they really do.

Most of them believe they are in the crest of the wave after going through this third transformation process. They are innovative, they has been able to reduce their time to market, they are gaining reputation within a variety of communities… They are not just Open Source consumers and producers any more. They are also contributors. Some of them even heavy and "successful" contributors.

But if you look closer, they have not adopted "the Open Source way".

This organization keeps their traditional processes intact. It is managed in the same way. Decision processes are taken like when it was a proprietary company, it has not improved transparency significantly, it does not share code and practises among departments.... there is a totally different reality in front and behind its firewall, between production and R&D, between management, engineering, customer support, etc..

This organization face friction because of this reality. It still cannot move fast enough. Upgrading is still too expensive since now they have to do it more often than years ago, upstreaming goes so slow, when it happens. They cannot control the communities they are investing on...

So going through a forth transformation becomes unavoidable. Some refer to this transformation as "upstream first".

 

4.- Upstream first or becoming a good Open Source citizen


This fourth transformation basically means that upstreaming is part of your development process, not an aside task. It also means that communities are part of your delivery strategy, not an after market topic, that R&D is a two way road where you do not just consume innovation created by others but you share yours, not just "promote it". You really need to get involved.

This organization will learn that by becoming more open, their engineers learn more and faster, so the organization itself. It is at this stage where the organization really understand where the real value is in the software they produce compared to what is commodity...or that is what its executives and managers most likely believe, once again. :-)

But open source (no capital letters any more) is not about being open, but about being transparent, which means that is not just about seeing what is behind the glass, but also understand it.

I believe the fictional organization I am talking about will have to take one more step, the fifth one. It will be about "becoming upstream".

 

5.- Becoming upstream or being an open source organization


This is about understanding that, if you consume, produce and contribute Open Source, the smart thing to do is becoming an open source company. I think it is naive to pretend taking full advantage of  Open Source while keeping your traditional corporate culture, which collides with the one of those who produces most of the software you consume and ship, who are your “upstream”. You are building your business on top of them. Since you cannot control them, become "them".

The smart thing to do is to surf the wave, not fight against it, generating friction. Any manager knows that friction is expensive, reduces focus and drives away talent.  It is bad for the business.

The required culture change to succeed in this fifth transformation involves thinking less about us (company and customers) and them (community), and more about us (ecosystem). It can't be any more about upstream and downstream but about technology and service. It has to be less about upgrading and more about updating, less about "manage" and more about "lead" at every level of the organization, not just referred to execs and managers.

It is a transformation in which engineers are empowered, where management is more focused on collecting information for execs instead of producing it, and after decisions are taken, their key focus is alignment. A transformation in which execs get closer to where the real value is, to people, because they are the "masters". A transformation in which engineers not just follow, they get exposed, they take responsibility and assume the consequences... getting paid for it.

An environment in which accessing to key information does not depend on your position within the organization chart, which means that power does not depend so much  on what others ignore, but decisions are taken based on shared knowledge. A culture in which transparency is the norm not the exception.

In summary, a transformation that leads to a stage in which the organization steers its ecosystem instead of driving it. So it leads it in a sustainable way.

 

A quimera?


I understand it might sound like a quimera, but:
  1. No more than it would have sounded 15 years ago any of the stories that so many CEOs or Open Source Program managers from leading corporations are telling nowadays in popular FOSS events about "their transformation".
  2. I do not think the debate is if this fifth transformation will be needed, but about when and how to go through it.
  3. My +10 years of experience in Open Source and +17 as manager tells me that, waiting to face any of the first four transformation processes until you have no choice is an unnecessary risk. I suspect the same will apply to the fifth one.
So my message is,
  1. Consume, produce and contribute to open source being a good citizen.
  2. Embrace Open Source culture... better sooner than later.

Pic link.

Thursday, May 12, 2016

Testing => quality. Really?

Introduction


Nowadays the topic automated testing is becoming mainstream. Organizations and projects are investing significant effort in creating tests, using tools to automate them and plug them in their delivery chain. Combined with continuous integration tools, automate testing increases the usefulness significantly. I obviously find this trend unavoidable. Sooner or later every software organization will eventually go through it, if they have not already.

This movement is fairly new. Concepts like automate testing or continuous testing, in the context of continuous delivery, still do not have 10 years of history. We need to be careful with trends. The topic is so hot these days that the association between automated testing and quality is becoming the norm, also in Open Source.

Open Source became the winning "culture" in several industries more than five or ten years ago. Automated testing in the context of continuous delivery was not popular back then. Still, Open Source influence and adoption expanded also because of superior quality.
How come?

When I think about quality in Open Source, one key principle and three actions come to my mind.

 

Principle: transparency


Transparency is about seeing what others are doing, but also about understanding. This second part is too often forgotten.

Action 1: Code review


Transparent code review, (again, see & understand) is, in my opinion, the most powerful quality assurance measure a project or organization can apply. It is the fundamental action in what some call the FLOSS development model.

It has a side effect that I really like as manager: it improves younger developers skills. It also brings many other positive side effects.

 

Action 2: dogfooding


A few weeks ago in a workshop with a customer, Codethink CEO Paul Sherwood was explaining this point with an example that I stopped talking about several years ago. I found it so obvious that at some point I gave up fighting for it. After listening to him, not anymore. The example was.... your organization is developing Linux based products, use Linux, not Windows.

Simple, right?

Dogfooding is another of those actions that in long term Open Source projects is frequently taken for granted but that is not the norm in commercial environment. So many projects driven by newcomers to Open Source do not pay enough attention to it.

The impact over quality of dogfooding in the mid term is impossible to calculate. Still I believe is huge.

Action 3: delivery model that maximises the influence of early adopters


Who are early adopters? They are the developers or power users who like to consume experimental or pre-releases of your "product". The number of those willing to report bugs is significantly bigger in relative numbers than in consumers.

Increasing the number of early adopters, reducing the hurdles they face to use your software, analyse/debug problems and report should be a key activity among those projects worried about quality assurance. Adapting your delivery process to maximise their impact, not just have a positive effect in the use cases your software was designed for, but in others, expanding the knowledge about how your software will behave in the hands of users. Like it should happen between developers and delivery engineers, the feedback loop with early adopters should be very short, so you can provide them improved pre-releases in short cycles.

Open Source has reached the current point understanding how important the role that early adopters play is.

Personal note about this third topic

I want to make a point here before moving forward.

It seems to me that there is a new wave of Open Source projects, specially those driven by commercial organizations, that underestimate the mid term effect early adopters have on the quality of a project. I also see how the Continuous Delivery hupe, focused on the developers and delivery engineers, is leaving the early adopters behind in some cases. Specially in those Open Source projects in which the project is developed and delivered by full time dedicated engineers.

Many projects pay little attention to making their frequent releases truly installable, documented, simple to debug without complicated tools or even centralised infrastructure, bug trackers simple/fast to use, treat bug reports as a valuable asset . In summary, early adopters cannot follow the pace and, when they do, they need to spend a lot of energy to be valuable.

Let's go back to the main argument.

 

Conclusion


Code review, dogfooding and early adopters in transparent environments has been, I believe, the pillars that has made Open Source what it is today in terms of quality. And then, only then, automate testing, or continuous testing comes to place, in addition, not in substitution, not before, not in between.... in addition.

Are you doing Open Source? Don't take shortcuts. Surf the "trend wave" instead of embrace it blindly. Learn first, look carefully what sustainable projects are doing.

Quality is as much as culture as it is about having a nice dashboard full of green lights. Testing => Quality is, in general, a wrong association of ideas.

And yes, test frameworks, board farms executing thousands of tests, green lights in dashboards, etc. are awesome. Probably a forth pillar in the coming future.

Monday, May 02, 2016

Say Hi! to the new GENIVI Development Platform

On Wednesday February 17th, the GENIVI Alliance released a QEMU image of the GENIVI Demo Platform ivi9 Beta version, together with everything needed (instructions, source code, recepies, etc.) to build GDP-ivi9 with Yocto. A few weeks later, on March 8th, the first release candidate was published.

Finally, last April 19th GDP-ivi9 was published targeting QEMU, Renesas Porter and RPi2. Check the release announcement and download the different images and source code from the GDP download page.

I joined the GDP project in November 2015, leading a small team of developers from Codethink with the idea of moving GDP from a demo platform towards a collaboration platform. In summary, going from +r-- to +rwx. 

What was GDP?


GENIVI Demo Platform was the compilation of middleware components developed by GENIVI integrated with Yocto or Baserock, based on poky, designed to showcase and test the work done by GENIVI's Expert Groups.

What is GENIVI Development Platform?


At GENIVI's 14th All Members Meeting (AMM) is was announced that GDP would change his name, from Demo Platform to Development Platform, reflecting the new spirit that has arisen during the delivery of the  GDP-ivi9 version.

The general idea will be to mature those GENIVI's modules that were developed as proof of concepts (PoC) and provide up to date software together with a SDK, to attract developers to participate as contributors, having GDP as their number one Open Source platform for automotive.

Find further information about GENIVI Development Platform at GENIVI's public wiki, in the GDP project pages. The name change, recently announced will be reflected in the wiki in the coming weeks. 

Coming actions


During the coming weeks, the GDP delivery team will focus on the following topics:
  • Migration from the current infrastructure to Github.
    • Confluence will remain as the project wiki and JIRA as the ticketing system. The same applies for the rest of GENIVI.
  • Add to our current targets another board: Intel Minnowboard
  • Define together with the GDP community the roadmap for the next GDP version.
  • Create a first alpha of the new version including the latest GENIVI software.
Feel free to propose enhancements or new features to GDP. The only thing you have to do is create a subtask under the ticket GDP-154, describe it and explain the benefits and potential risks/challenges. We will discuss them through the mailing list. I am looking forward of seeing Plasma 5 as part of GDP.

GENIVI 14th AMM and other events to promote GDP.


After te release of the new version, GDP maintainers and myself have been concentrated in making sure GDP was ready for  GENIVI's 14th All Members Meeting (AMM), that took place in Paris from April 25th to 29th.

I participated as speaker in 3 sessions and my colleagues at Codethink delivered a couple of Hands on Sessions about GDE-ivi9. It has been a lot of work but a good finish line for this release cycle. We will publish the slides the coming days.

A few weeks earlier I presented the GDP project at the Embedded Linux Conference (ELC), that took place in San Diego from April 4th to 6th. It was my first time at this conference and I enjoyed it. I also participated at the Collaboration Summit, invited by AGL and the Linux Foundation. I will provide some more details about these events in a later post.

I plan to attend to QtCon to promote GDP among Qt/KDE developers and to the Automotive Linux Summit, that will take place in Japan, to spread the word about this open project for automotive. I have also confirmed my presence in June 2nd at the OpenExpo, in Madrid. It will be my first event in Spain in quite some time.

Summary


It has been a very busy 6 months but very productive. Leading a small but promising Open Source project, that might have a big influence within automotive in the future, working together with my colleagues at Codethink and GDP community members, has been very interesting. I am learning a lot about this industry...by doing.  

Wednesday, August 26, 2015

Virtue of Necessity. Canary, sublime your company.

The past July 16th I participated in the Tenerife LAN Party, in its section Tenerife Innova, invited by the Free Software Office of La Laguna University, included in the track titled (free translation from Spanish) Open Source from the Canary Islands, stories told in First Person.

This Free Software Office is well known in Spain for managing the biggest KDE deployment in Spain with 3k computers spread in several computer labs, laboratories and libraries, among other internal projects.

My talk (20 min) had as title something like: Virtue of Necessity. Canary, sublime your company. You can find the slides (in Spanish) in my site or in my Slideshare account.

What was the talk about?

The natural growth path for a software company is creating a site, grow until it reaches a point in which, a second production site is needed. Meanwhile, departments like sales and technical support might grow distributed. As the company grows, the number of production sites grow  with it. The organization structure vary with the nature of the company, sometime production teams are replicated across different sites, sometimes different business units are divided per site and others a particular site host teams that take care of different products-(micro)services.

In general, if the company follows an "agile" approach, it will try to reduce the inter-sites communication needs by placing the team members together in a particular site. Based on my experience and how FOSS has been developed, depending on the market you are playing, turning your company into a distributed environment might be a smart move. 

Let's start providing some context and definitions.

What do I mean by sublimation in this context?

Sublimation is a state change in which a substance transit from the solid state to the gas state without going through the liquid one.

What do I mean by a distributed environment?

In most agile literature, in fact in most software development management books, distributed environments are multi-site distribution. But in Open Source, we refer to a different environment. I have came out with the following (subjective) definition:

A distributed environment/organization:
  • Has no site with a significant number of employees (developers). Most of them work remotely.
  • Access most (if not all) corporate application though web, not through VPN, that is, are WAN environments, not LAN.
  • Uses chat-IRC (or equivalent) and video conferences as the default synchronous communication channel.
  • Employees spread across several time zones.
  • Multicultural environment, having English as the default language.
  • Organize regular face to face sprints (as we understand it in FOSS=hackathons), maybe even a corporate event where the whole company or business unit get together.

Open Source as distributed environment

Free/Open Source Software has a geographically distributed nature. As you know, most of the relevant communities, no matter which size are we talking about, are formed by developers located in many different countries, working from home or a company site. If we take a look at the most relevant ones, they are truly global. Every tool, every process, has been designed (intentionally or not) having in mind this distributed nature.

Now that Open Source is everywhere, more and more companies are embracing it, participating on its development, collaborating in global communities. from the process perspective, they are being influenced by the Open Source way, including its distributed nature.

Many of the companies that are embracing Open Source are understanding that adapting to this new environment makes collaboration easier. It reduces the friction between community and internal processes. There is a long way to go but I believe it is unstoppable for a variety of reasons (out of the scope of the talk). We are starting to see more and more organization that are fully distributed and start-ups that are born with such structure in mind.

Canary Islands, a fragmented market.

The Canary Islands is a group of 7 islands in the Atlantic sea, with two major ones (900k people each) and 5 smaller, for a total of 2.1 million people and 11 million tourists per year. Obviously tourism is the main industry, so there are 6 international airports, two national ones and 10 harbours, half of which regularly receive big ships/cruises.

Data connectivity has improved a lot the last 10 years but, due to the difficult geography, it is unequally distributed across the islands. Even within the main islands, Tenerife and Gran Canaria, there is a significant percentage of surface with zero internet coverage.

So it is a very fragmented market and, although with first class communication infrastructure, travel across the islands takes a significant amount of time, it is expensive and connectivity might be a challenge. In general, the transportation strategy has been designed to bring people from Europe not for internal mobility.

This means that, as a software company, consolidation/growth in such market is tough, very tough, even if you focus on tourism.

Software companies there expand following the "natural" approach, which is by creating a software production centre in one of the main islands, providing support from there to the other islands. Until you consolidate your position, software companies cannot afford to have developers/technical support in the second main island. If service/support is required in one of the small islands, you simply travel there. The limitations that software companies has to face due to the market conditions, rarely allow you to create a second software development centre in the Canary Islands.

There are very few Spanish cities with daily direct planes from/to Canary Islands throughout the year. Madrid and Barcelona are the biggest markets but also the most expensive cities. The flight takes 2:30 hours to Madrid and 3 hours to Barcelona, which is a lot for European standards. So opening a second development centre in the mainland keeping the headquarters in the Canary Islands is a real challenge.

In other words, if you want to scale your business, you need to assume bigger risks than companies based in the continent, despite being a cheaper place and having plenty of professional due to the existence of two Universities.

... but,

In my talk I tried to show that all those limitations can be turned into advantages if organizations, early in their consolidation process, or even from the very beginning, adopt a distributed approach. These constrains offer a first class laboratory to experiment with some of the key variables that need to be managed when scaling up your company, while leaving aside some of the most complicated ones, related to great extend with the internationalization of the organization.

I made a call to sublimate your company, going from an "on-site" to a fully distributed state, ignoring the multi-site state. Even better, create your software company as a distributed environment since the very beginning.

Why sublimating your company in the Canary Islands?

I summarized the advantages of sublimating your company if you are based in the Canary Islands, Spain, in the following statements:
  • Distributed environments adapt better to the Canary Islands environment.
  • It will prepare you earlier for the internationalization phase, keeping a smaller size.
  • Distributed environments adapt well to certain business models and support needs, that are becoming popular nowadays.
  • If your company want or is already heavily involved in Open Source communities, adapting your internal processes to those collaborative environment you participate on is easier.
  • Talent attraction is less difficult.
  • Some of your fixed costs turn into production costs, that is, you gain flexibility.

Which variables will be affected by subliming your company?

These are the most relevant variables to consider:
  • Cost per employee: reduction of fixed costs per employee. Increase of travel costs.
  • Organization chart: from vertical to horizontal
  • Human Resources policies: training, coaching, people management. From f2f to online.
  • Data privacy and security: adapt to a WAN environment.
  • Tools: adapted to distributed with higher latency environments
  • Schedule / availability: culture shift, from presence to availability
  • Development and support methodologies: from f2f conversations to remote synchronous/asynchronous channels. From agile to FOSS? Is FOSS agile?. From 8-10 hours or rotation to a higher daily production/availability window.
  • Costumers relation/engagement: from tradition account management to "community" engagement management. Engineers interface customers.
  • Transportation and connectivity: employees need to be connected and travelling demands will increase. Accountability/reimbursement processes will be more complex.
  • Business model: your business model might need to adapt to your new distributed nature.
  • Potential market: the presence of employees in new areas and the fact that your processes are adapted to distributed environments might change your target market and/or nature/size of potential customers.
  • Competitors: the influence of your new distributed nature might alter your positioning against competitors.

There are more but these ones are the ones that should be considered carefully before subliming your company in the Canary Islands. As you grow, internationalization will knock at your door very soon. There are other variables to consider in that case. They are not the scope of this talk:
  • Time zones
  • Multiculturalism
  • Language barrier
  • Retributions and incentives. Cost of living.
  • Fiscal and employment legislation differences
  • Travel and accommodation costs and reimbursements
  • Accountability and taxes
  • Many more...

Summary

The Canary Islands is a tougher market than the mainland of Spain. Adopting early in the software company life cycle a distributed nature allow you to adapt better and faster to this environment, preparing you better for later stages too, specially the internationalization phase. Sublimation provides you a competitive advantage, specially if you develop Open Source and participate in open collaborative environments.

There are a number of variables that should be carefully considered though. Managing them correctly is a requirement to succeed.

Friday, June 12, 2015

openSUSE transformation step 2. The user oriented distro.

openSUSE Tumbleweed is a rolling release that was designed with a clear goal, target and metric. It was developed following a clear picture of where to go. Design it was a painful but unavoidable process that challenged an assumption established in many people mindset back in 2012. Latest/greatest and stability were incompatible. Hence, Factory, the rolling release back then, was for hardcore SUSE/openSUSE OS developers only. As rolling distros, there were more popular and better options out there.

This summary might help you to get some background.

Goals are easier to achieve if you have a good reference to beat. For those who worked in the project, Gentoo and specially Arch Linux were those references. As you can imagine, transform openSUSE required management support. We had it, specially from Roland Haidl, Operations and Communities Director at SUSE back then. He created the environment that allowed those who worked in his department to be creative.... and take risks.

Simplifying, for the new "development version", a.k.a Tumbleweed (former Factory), the goal was to implement a model that allowed us to improve the existing Factory one, based on continuous delivery. The target chosen were our core contributors (packagers fundamentally) and the metric was, in summary, to make sure that, no matter how wrong things could go after an update, you would always have a console and network, so you would be able to revert your change. In terms of the process, the resulting integration deployment processes should be transparent, not just internally but also from our community members perspective. It also needed to be simpler in order to gain contributors, not just users. And it needed to empower them to own it.

Instead of following what SUSE was doing back then, the company dedicated resources to challenge itself. As result, openSUSE Tumbleweed is today, not just the best rolling distro out there, with all what that means in terms of excitement among its contributors, but is generating higher value to SUSE, since the company have an outstanding playground at home that allows them to incorporate true innovation into their production process before their competitors do.

openSUSE is discussing nowadays to take a second step, this time focused on its user oriented version. Today is openSUSE 13.2.

In my opinion, based on the previous experience, and independently of the decision/discussion process chosen, the same steps need to be taken. They are unavoidable in any transformation process. It is necessary to define a clear goal, something short that you can explain and understand easily, a clear target and a key metric that helps to clarify the "acceptance criteria" to be used during the whole process.

Like back then, I would like to see SUSE challenging itself, putting in question well established principles within the OS industry. Again, choosing a reference would make the final picture easier to achieve.

Most openSUSE users are desktop users and sysadmin. If, as I conclude from the latest oSC15 videos and factory mailing list discussions, sysadmins are the chosen target, It would be great to see SUSE/openSUSE challenging the assumption that, through a continuous delivery process, you cannot release a stable and high quality (for the target) distribution. That stability is only achievable through a waterfall like model. I would choose CoreOS as reference. It is a project that, based on different questions, is providing innovative answers to new challenges.

I would like to see that, base on the current process (standing on the shoulders of giants) openSUSE/SUSE creates a process that "pulverize" the current mindset, deprecating many of the existing problems, focusing on solving new ones. Imagine the best of both worlds, a new paradigm of OS with the green values.

It took about a year and a half for a dedicated team to release what today is Tumbleweed. I think that this second challenge is bigger than the first one. An even bigger commitment from SUSE will be needed in order to succeed.

But if the resources are there, the creative environment is set, the right steps are followed and the openSUSE community supports the effort, there is nothing that can stop the project to achieve what today are dreams. SUSE has the talent, and the experience, to make it happen.

I wish them all the best in this new challenge.

Monday, May 18, 2015

Bitacora: impact mapping

Introduction


This is the fourth of a series of articles about Bitacora. Please read the previous ones to get full context:
  1. The diary (aka bitácora): towards alignment in distributed environment.
  2. Bitacora: environment definition.
  3. Bitacora: personas 

Why Impact Mapping?

The steps taken so far are standard for me since long time ago. At this point though, depending on the nature of the product, I did not have a fixed procedure to follow until the creation of user stories. At some point I ended up creating use cases and relating them through mind maps.

A few months ago I read Impact Mapping, from Gojko Adzic, a reputed Agile consultant and writer. He proposes a method to go from business model to Epic/stories description that I found interesting. It fills my gap with a systematic approach using a tool I love, mind maps.

It also helps deprecating a common challenge with use cases. When you get dirty with them, it is easy to write many that are not key for the main goal of the tool. Once you have many of them, it might become tough to relate and prioritize them.

Gojko propose to get rid of use cases and go directly from impact mapping to describing the epics/user stories. I have not been able to do so. It might be that I am simply too hooked to use cases and it is only a matter of time before I follow exactly the new process. It can be also that impact mapping is rigid compared to use cases so you can only get rid of them in a second/third iteration of the impact mapping process...

In any case, for Bitacora I started with an impact mapping. Once it was mature enough, I started from scratch doing the use cases. My impact mapping ended up containing 75% percent of the use cases.... and already structured. I assume that, in a collaborative design process, this percentage will be greater. I assume also I can get better with practice. So I am confident to adopt Impact Mapping exclusively. I might loose some use cases but I gain structure. Relevant but at this point ignored use cases will probably pop up later in the design process or during development. Mistakes in the prioritization due to structure deficiencies has a bigger (negative) impact along the product life cycle.


What is an Impact Mapping?



Mr. Adzic describes in his book Impact Mapping as "a mind-map grown during a discussion facilitated by answering the following four questions: Why?, Who?, How?, What?"
  1. Why: centre of the map, represent the goal of our application. It should include the desired achievemnt based on a key metric.
  2. Who: first level of the mind map, it refers to actors who can influence the outcome. These are the personas. At this point of the process, it is not mandatory to describe them in detail
  3. How: second level of the mind map, it refers to impact we are trying to create on the actors, that is, how the product change their behaviours.
  4. What: third level of the mind map, it refers to the scope, that is, what can we do to achieve the desired impact. Depending on the dimension and complexity of your product/service, this third level will be close or far away from a user story. You can either extend the What section into sub-levels (product approach) or define epics (scrum of scrums). 
You can get further information and diagrams in the website impactmapping.org I am still unsure about the direct relation between the What section and what engineers understand as a derivable. I prefer to reserve derivable as concept to a later stage (user stories) so they are analysed in detail by the product team, together with acceptance criteria. I find this arguable though.

Bitacora Impact mapping

Once you:
you should be able to understand the coming Bitacora Impact Map.

If not, I have probably done something wrong, which wouldn't surprise me :-) or you are not familiar with the problem to solve. Some of the ideas like "tag page" will be described in the user stories, so don't panic, we will get there.

Since these graphs has gone through several review processes, it might happen that the partial pics are not 100% up to date. The general impact map should be though.

Center and Level 1: Why? and Who?


The center lacks the quantitative goal. Since this is an ideal exercise, I will ignore it for now. We will go down to metrics and measuring impact at the very end.

At the end of the previous article, about personas, I wrote that for a introductory analysis, we could reduce them to 5. Since I went further, I considered groups and described personas for each group since we will use them at some point.

Level 2: How?

This is the second level that defines the impact for the product team members.


 This is the second level that defines the impact for the other personas.




 As you can see, the impact is described in the following areas:
  • Participation: the tool should promote an increase of interactions based on what each persona did.
  • Team/project follow up: bitacora should make following up a team or project activity easier.
  • Analysis: analize what the team is doing or has done should be easier with Bitacora. Self analysis is as important.
  • Report: Bitacora is designed to improve reporting in bothe, vertical and horizontal direction (matrix).
  • For Team Member, Bitacora should work as the central point for everyday (short term)  information.

Level 3: What?

For a easier visualization I have divided this level in three different images. The first one correspond to Product Team:



This is the third level for the Scrum Master persona:




 And this is the third image, corresponding to the other three groups:


Complete Impact Map

Here you have the complete map.

You might be wondering what are the icons for.

I mentioned earlier that one of the advantages of using Impact mappings is the structure. As a consequence, you can start to assign priorities to the different scopes which is very useful before writing down the User stories. 

Note:

The result of this process should in general be features, but in certain circumstances, at this point it could also be processes. I used the "help" icon to reflect them. his concept is arguable. I am at this point unsure if it will survive the story description.

Series of articles

  1. The diary (aka bitácora): towards alignment in distributed environment.
  2. Bitacora: environment definition.
  3. Bitacora: personas
  4. Bitacora: Impact mapping
The following article will describe the use cases I worked with so you can compare both approaches.

Sunday, May 17, 2015

Bitacora: personas

Introduction

This is the third of a series of articles about Bitacora. Please read the previous ones to get full context:
  1. The diary (aka bitácora): towards alignment in distributed environment.
  2. Bitacora: environment definition.

The environment description is incomplete without the personas. I like this concept for many reasons but the main one is because I get extremely annoyed by the use of the word user.

When designing a product/service, it is hard to agree on what a user is, specially in engineering environments. In marketing/product environments, in general is in the DNA of those involved in product design to differentiate buyers from consumers (users). Market segmentation is understood by default. In my experience, this is not always the case in engineering environments. 

I must admit that I do not use this concept every time since again, in my experience, engineers tend to show a high resistance to this concept, specially those who have direct contact to users, that is, those who work in open and collaborative environments. They tend to consider it mainly a simplification not of a representation.

Groups

We will assume there are 4 groups/roles involved in our scenario. Each group will be represented/formed by one or several personas. To label the groups, I will use names associated with roles you probably are familiar with, which is not a good practice since it limits their potential translation to a different scenarios. I do it so it becomes easier for you to identify them. The groups, based on its interaction with Bitacora are:
  1. Product team: professionals included in the team that will develop the product.
  2. Customer representative: professionals in charge of the requirements/backlog. Responsible for the product.
  3. Senior manager: professional in charge of the above groups.
  4. Supporting actors: people that interact with the team and are needed to develop/launch the product. They can be part of the company, the open source community that develops the technology the product is based upon or the sponsor.
If this blog series would try to reflect the Bitacora design process sequentially, the number of personas, even groups, should be lower, increasing them when necessary during the design or development process.

Product team


 I have considered 4 personas:
  •  Junior developer (jdev)
  • Senior Developer (sdev)
  • Artist (art)
  • Scrum master (smast)
persona_kareem
persona_james
persona_lisa
persona_byron
Artist represent non-technical profiles technical teams might include like domain experts, communication experts, etc. At this stage, defining 4 personas is not the right thing to do. Two (scrum master and the rest) would have been enough. only when going into details, the four personas will make sense.

Customer representative


I have considered two personas:
  • Product owner (pdm)
  • Sponsor (spon)

persona_earvin

persona_ac

Like in the previous case, one persona is enough for most of the bitacora design process.

Senior manager



I have considered a single persona, the engineering Vice President, which is the most common non-executive senior manager (EVP).
persona_michael

 

Contributor


I have considered three personas:
  • Marketeer (mark)
  • FLOSS community member (cmem)
  • HR representative (hrep)
persona_candece
persona_kurt
persona_tina

The Marketeer could also be any other professional within the company that interacts with the team. The FLOSS community member could also be an external consultant/domain expert contracted to help the team.


As in the previous cases, for most initial stages of the process we will follow, the first two personas could be considered as one. The third one has a very specific and limited role in this design, based on my experience, so it could be ignored or somehow included in the senior manager persona. I will refer to it in very few (but relevant) cases.

For a close follow up you might want to download the slides.

The cliparts has been taken from clker.com and 1001freedownloads.com If

Conclusion

As you can see, I have considered 11 personas, which is a lot at this point. But I started with 5 which must be considered the main ones:
  1. Engineer
  2. Scrum master
  3. Product owner
  4. EVP
  5. FLOSS community member.
So from now own I will refer to them by their names or roles. Based on your input and the process design development I might refine them.

Series of articles

  1. The diary (aka bitácora): towards alignment in distributed environment.
  2. Bitacora: environment definition.
  3. Bitacora: personas
  4. Bitacora: Impact mapping
The following one will describe the impact mapping. This section will be updated with the coming articles.

Saturday, May 16, 2015

Bitacora: environment definition

Introduction

This is the second of a series of articles about Bitacora. Please read the first one to get full context.

In order to understand the perfect implementation of a Bitacora we need to go down the road of a product definition. One of the first steps is to define the environment.

Bitacora is specially designed, although not exclusively, to distributed teams. So we need to first understand what do we mean by a distributed team.

Technically speaking, we need to define the environment in which our product will live. At this point, we will take it as a hypotheses, which should be the result of studying and defining our business model. But that is out of my scope so I will use as starting point an abstraction of the different environments in which I have worked. In most of them I have worked with imperfect/limited/working versions of (the ideal) Bitacora. 

Description of the environment

We will assume the Bitacora is designed to be used in a professional environment, a company that has four offices:
  • The headquarters are in Silicon Valley (UTC-8).
  • The main engineering office is in New York, US (UTC-5).
  • QA and services dept. are based in Hyderabad, India (UTC+5:30).
  • The fastest growing office is the London, UK (UTC).
Three years ago, the company changed its growth strategy due to the difficulties they were facing to attract talent. The increasing relation with some Open Source communities acted as catalyst for this change. So now, instead of relocating people, they hire the talent wherever is available, becoming what I refer to as a distributed environment.

50% of the people hired (not including sales and customer support) the last three years are home based. They expect to increase this number up to 75% in the coming three years.

The company was originally conceived as an agile company. Every product/service has been developed following Kanban or Scrum. One of the challenges they are currently facing is how to adapt those methodologies to:
  • The increasing relation with open collaborative environments.
  • The distributed nature of the teams.
At the same time, the company need to face an increase of the services associated to the products they are developing. This is creating a friction between processes associated to product development and services.

Due to the increasing interaction with open source communities, it is expected that this challenge will get more complicate due to the R&D nature of the work done in these communities, that incorporates new and different processes that the company do not has control upon.

But above all, the main challenge brought by the new distributed nature detected by senior managers is the the reduction of alignment within the company which has two clear symptoms:
  1. Appearance of silos.
  2. Complains by senior engineers and first level managers of lack of big picture and reduced horizontal communication among teams.
A new team has been formed within the company to develop a new product, which is an evolution of a former one, based on a technology developed by an open source community. The new product is sponsored by a reputed German company that is building a complete solution for a vehicle industry leader.

This team is multidisciplinary and, following the nature of the company, heavily distributed.  It will have direct relation with the open source community the product is based on, the sponsor and several key stakeholders within the company.

Finally, since this is product considered key for the future of the company, senior management want to evaluate the work done on a regular basis.

You, reader, and I, are in charge of, using this team as test case, designing a tool that improve alignment keeping high levels of autonomy, using agile principles while reducing the impact on existing company processes to the minimum.

That tool is Bitacora.

Why this environment?

I see the above environment as the maximum common denominator of the key ones I have worked in, that is:
  • Team distributed in several offices in the same time zone
  • Team centralized in one office with a significant % of remote (home) work with a remote product manager, all in the same time zone.
  • Team distributed in two offices and several remote team members in different time zones. 
  • Team completely distributed (no office) on two correlative time zones.
  • Team completely distributed across the globe in several time zones.
The possible combinations increase if we introduce culture/origin/native language/profile of the team members or sponsors, the nature of the company/product/service being developed, my role in the team and the relation of the product with collaborative environments. 

I also wanted to choose an environment that might sound familiar to my blog readers. This scenario is probably not the one where Bitacora would have the highest impact, but again, my main goal is explaining the ideal solution, not creating the best business plan.

Series of articles

  1. The diary (aka bitácora): towards alignment in distributed environment.
  2. Bitacora: environment definition.
  3. Bitacora: personas
  4. Bitacora: Impact mapping
The following one will define the personas. This section will be updated with the coming articles.