Here on August 1st, I am joining the MariaDB Foundation in a position as Chief Architect Replication.
Who am I? I am a long term MariaDB developer; in fact I was one of the very first to join Monty in starting MariaDB back in 2009 (I think Sanja may have beaten me by a few days). Over the years I have done many things on MariaDB, and especially in replication, where I architected and implemented the main new (compared to MySQL) replication development in group commit, global transaction ID, and parallel replication. I gave a talk last year at
the MariaDB Server Fest 2023 where I explain some of this work: https://mariadb.org/replication-dirty-secrets/ .
So, in a sense this is going to be very much “business as usual” for me. I was already working on MariaDB development more or less full-time, and will continue doing so in this role with the Foundation. But I am hopeful that the MariaDB Foundation will provide me with a stronger platform for continuing my work. Formalising my position as the project replication lead will clarify my role as the development community grows, and should help avoid me getting kept out of important decisions in the project.
Most of all, I am hoping to use my new role as a way to interact more directly with the big users of MariaDB, some of who are sponsors (https://mariadb.org/about/#sponsors) of the MariaDB Foundation. Seeing the actual use-cases and understanding the day-to-day challenges faced by big users provides inspiration and important insights for the long-term planning of future replication development. And it is always a pleasure to work with
top skilled application developers and devops, who provide invaluable feedback; for example in verifying performance of new development and tracking down tricky bugs.
For the Foundation, I believe adding a core developer to their team is a great boost to their position in the developer community and gains it credibility in the greater MariaDB community. The Foundation’s success in attracting sponsors is most impressive, and I think there is a unique opportunity here going forward.
As a non-profit, the MariaDB Foundation can provide the most skilled and experienced developers with the support needed to concentrate on the long-term development of the architecture and code. This in turn will show to the existing (and new) sponsors the value of having an independent and strong player in the development community. So I hope that this is the start of the Foundation focusing on adding more core developers to their team.
Speaking of long-term development, I am working on several such ideas of my own in replication. One is a new binlog format which is stored transactionally by a storage engine (ie. InnoDB); this will avoid the need for two-phase commit and significantly improve scalability, and provide numerous other benefits as well. Another idea is an overhaul of the parallel replication thread scheduler, which is currently very naive and mostly unchanged since the first version. These and other ideas of mine deserve a longer story at another time.
And on a boring, but I believe crucial note, I want to continue working on improving the robustness of replication and avoiding regressions on upgrades. What I have seen working with users is that, more than new features or even performance improvements, they need their replication setup to not break at random intervals, and their MariaDB upgrades to not
introduce bugs that affect their application. I believe users have a right to expect their database to “just work”, and doing this for something as complex as MariaDB replication is a tremendous challenge. Again, as a non-profit, I think the Foundation has a key role here, relieved of the day-to-day pressure from product management and marketing to release (too) quickly.
So in summary, my plan as new Chief Architect Replication with the MariaDB Foundation is to try to spend a significant amount of my time on long-term replication development projects and code reviews of replication patches. These are tasks that require considerable efforts and focus, and the challenge for me will be to keep that focus among the myriad small day-to-day tasks that are also bound to pop up all the time.
Looking forward to working with you all in the MariaDB community at large!