Status 3 Oct 2016
New approach necessary to overcome volatility of upstream sources. Reconsidering overhead of Chef approach. Jenkins 2 is not behaving with automated install, so an older version is still in use. Currently working on Ubuntu platform.
This is a project to create a simplified, reproduceable DevOps pipeline for educational purposes. It uses "infrastructure as code" principles to configure git, Vagrant, Java, JUnit, Ant, Jenkins, Chef Zero, and Artifactory into an integrated, end to end system.
There are a number of benefits you might find in this project. While the basic DevOps principles it illustrates are simple and widely understood, it can nevertheless be frustrating to properly configure all the interactions in an end to end DevOps pipeline. There are any number of issues lurking in integrating the pipeline: .ssh setup, permissions, build choreography interactions (e.g. git/Jenkins/Artifactory), software versions, and the like.
This gives you a functioning starting point, a "known good" baseline running as a cluster of 6 Ubuntu VMs under Vagrant and Chef, that does the following:
- Gives you a basic test-driven "Hello World" Java example with JUnit, Ant, Tomcat, and git.
- When you execute a build on that environment, if successful it pushes to a remote master git repository.
- That commit then triggers Jenkins to execute a build on a "slave" node.
- If that build succeeds, the result is archived to Artifactory.
- You then can (with Chef) deploy the result to a simulated "production" node.
All the configurations are expressed in the Vagrantfile and the Chef cookbooks, so you can inspect and adapt them. No magic here. See the instructions.
Some may be particularly interested in the Calavera example of how Chef can provision Jenkins through the Jenkins api. This includes provisioning and controlling a slave, integrating with git through a githook, and integrating with Artifactory. It's all there; have a go with it. The Vagrant machine that runs Jenkins is called "hombros." See the hombros cookbook and the brazos cookbook.
See https://github.com/CharlesTBetz/Calavera/wiki/Calavera-Home and other wiki pages for full, evolving description.
See http://www.lean4it.com/2014/10/devops-simulation-for-education.html for initial motivation.
This has been a very active fall. The biggest change has been the creation of a viable multi-tenant architecture. It's still a bit kludgy, but with the VagantfileTemplate and the template.sh script the simulation can support 4 teams now (each with their own pipeline) and could easily scale higher.
I am not sure if there have been networking improvements in Vagrant, but issues I was seeing earlier in the year with VMs disappearing seem to have stabilized. I am looking forward to the next 3 weeks of labs, which will be based on the new platform. The only lingering question I have is how well my poor server will perform with 24 Vagrant VMs on it, but nominally it should be OK, as it has 48 gb of RAM. (update: it's performing fine.)
See lab 04 and (when done) labs 5 and 6 as well for the platform application. The labs may be a little sparse at first, because so much time went into the platform. I also don't have time to document the exact usage of the templating solution for now. (./template.sh xx where xx=1-4, creates a Vagrantfile for that directory with machines numbered accordingly. Some sed action to search/replace machine names in the Chef resources. Yes, could do this more elegantly with Chef.)
Struggled with Artifactory 4 and eventually reverted to 3.51, but we will move forward again.
Open for business. Please let me know what you think. And please help.
2015-03-02 note OK I changed my mind about releasing. After I finished the installation instructions yesterday and confirmed a couple succesful builds on different machines, I decided to go ahead and call 0.3 the first official alpha release.
The last 3 weeks have marked Calavera's debut in a classroom setting. It has been a lot of work and very exciting for both me and (I think) the students. This week, they are standing up their own instances of the Manos development environment.
Been doing some final polishing. The public Calavera alpha release is delayed due to my educational commitments - have to prioritize creating the labs. Testing the system under fire in the classroom also seems appropriate before publishing.
But a motivated person can certainly download and stand up the virtual machines at this point. I am currently working on the installation instructions.
See previous updates on the Calavera blog
The next major steps will be:
- Create a MEAN-stack based development pipeline
- Support Docker in addition to VirtualBox (may deprecate VirtualBox, depending)
- Integrate with Cloud providers
I'm an architect, advisor, and instructor. My career focus has been the "business of IT" including concerns such as enterprise architecture, IT service management, IT portfolio management, and IT financial management. However, I also believe that hands on engagement is essential.
I am not a professional software developer, infrastructure engineer, or systems administrator, although I know many. I have the deepest admiration for the professionals I see here on Github developing Vagrant plugins, Chef cookbooks, and the like. I know my work is not up to that standard. There are any number of aspects in these scripts that professionals might criticize. In part, this project has helped me learn the technologies at hand, so it is by definition amateurish.
However, as near as I can tell the concept is original, and I intend to build on it far beyond a DevOps pipeline. As part of the InsanIT initiative, it realizes the architecture principles I am using in my 3rd edition of The Architecture of IT Management. It is also a reference implementation of IT4IT. More to come on this.
I welcome collaborators and am ready to entertain pull requests if anyone wants to help. This work is a life priority for me.
One audience that I hope will find benefit in this is people like me - mid-career types who are watching all the buzz about DevOps in the media and looking for some accessible way to get a little deeper into it. Folks, when I started this I did not know ANY of the technologies here except a little bit of Java.
You can of course take 2 week courses in each of these technologies, but this is an alternate approach: see just enough of each in action to understand how it contributes to the overall system. It's been well worth it for me.
See the wiki for further information. And of course www.lean4it.com.