The Problem: Continuous Integration and Deployment
Previously in this blog series, we discussed how DX improves custom Apex development and how it helps manage changes in source of truth. In our final installment, we will combine these features to discuss how DX changed the Salesforce deployment lifecycle.
Salesforce has historically relied on change sets to manage deployment between environments. For small enough changes, change sets were no problem; but as Salesforce has grown, so has the amount of custom development that has gone into it. What was once “happy soup” is now just a pain to manage. With this increase in custom changes, the inability to automate change sets has become a glaring inefficiency. This method requires that a user manually keep track of every change made, create a change set, populate the change set with all of those changes, and deploy the change set to the target org. There is not even an ability to import a list of files, objects, fields, etc. to speed up the process. This issue becomes even more complicated when there are multiple developers working on the same instance, as it is difficult to coordinate change sets between concurrent workstreams. And while Salesforce has made some improvements to the process in the last few years (running a “Check for Dependencies” action to assist), it remains slow, tedious, and antiquated, as it relies on another org for changes rather than a source of truth.
With the introduction of the DX CLI, Salesforce has opened up the possibility of automating deployment. Because Salesforce functionality can be performed via the command line (most importantly deploying source code to an org), these commands can be scripted and executed from a variety of CI/CD tools (Jenkins, TeamCity, BitBucket Pipelines, etc.). Salesforce has already published a great tutorial on this that can be found here. This ability allows one to automatically deploy source code from a remote repository via a CI/CD tool, rather than manually creating a change set or using DX to deploy from a local machine. With the combination of CLI and source of truth via DX, Salesforce is moving away from manual deployment and taking a step towards continuous deployment.
To further assist with CI/CD, Salesforce has introduced packaging. In our previous blog post, we briefly mentioned second-generation packaging as a tool that can be used to help transition from org-based development to artifact-based development. The idea behind artifact-based development is to move toward a more modular development approach — rather than managing all changes for a large org in one place, code and customizations can be split into smaller bodies of work that correspond with specific business goals and development teams. It’s easy to conceptualize breaking up source code and metadata into smaller projects, and packages provide the means to do just that.
There are two types of packages: managed and unlocked. Managed packages are not a new concept, and the second generation is not all that different from the first; they are generally used for creating applications that can be sold to Salesforce customers. Unlocked packages are created and managed by the development team, it’s easy to tell which packages components come from, and code and metadata contained in the package can be changed on the fly in the org where it is installed. When source is updated, developers can create and release a new version of the package that can be installed in scratch orgs, sandboxes, and production environments. Versioning, releasing, and installing packages can all be done with the CLI and can be scripted for use with CI/CD tools, so the option exists to deploy source code directly or to use packages.
Because Salesforce orgs can be very large, it is not unusual to have overlap between projects, especially when it comes to permissions, custom fields and objects, layouts, etc. To help manage that overlap, unlocked packages can be made dependent on other packages — both managed and unlocked. This allows for shared code and metadata to be managed in a single package used by multiple projects and teams. As long as packages are installed after the ones they are dependent on, it is still a relatively simple feat to manage deployment with CI/CD tools.
As we alluded to in our first installment of the series, the problem of conflict resolution still remains. Though we are now able to track changes in source control, and deploy these changes automatically, Salesforce continues to have a “last in wins” mentality: whoever deploys last wins. This is not an issue if everyone is on the same page; however, in the transition to DX, this becomes problematic. If developers are working with DX while administrators are using traditional methods, problems arise. Code is overwritten, objects are changed, fields disappear, and there is no way to track who did what because only some of the changes are managed in source control. If communication is clear during the transition, it is possible to mitigate these problems; but even then, it is still likely that something will be lost in translation.
For example, you can train administrators not to change code or remove fields, but what if they add a field and forget to write it down, capture it in source, or tell a developer? Life will go on, changes will be made, functionality will be added, but it is likely that this change will be forgotten and, most importantly, not captured via source control or change set. When the sandbox where this change was introduced is refreshed, the field will be wiped out and chaos will ensue. While this may seem unlikely (who could forget something so important?), it is a problem we frequently deal with in heavily customized orgs. In order to avoid this issue entirely, it is important that the transition to DX be made swiftly and universally, so that everyone is working the same way. This does not mean that an org has to be instantly converted to DX — just that all users must know how and when to use DX in order to make changes.
Lastly, though second-generation packaging improves the development lifecycle in Salesforce, it is still a beta feature, and sometimes it shows. For one thing, beta features don’t receive any Salesforce support, so if issues are encountered, the only way around them is to investigate on your own. We have personally seen some very cryptic and even empty error messages, and dependencies don’t always come together cleanly. Fortunately, dependency management with second-gen packages is continuing to improve, especially since the development of the Force-DI framework.
Modern Era of Custom Salesforce Development
In the last year, Salesforce has come a long way in how it approaches custom development. They have vastly improved their tooling, allowing a developer to write with Apex like they would any other server-side language. The introduction of the scratch org allowed for improved metadata management and opened up the door to a controller source of truth for Salesforce orgs. And to bring it all together, the CLI allows an organization to implement effective CI/CD and replace time-expensive change sets, creating a smooth experience from development to deployment. With the release and continuing improvement of SFDX, Salesforce has stepped into the modern era of custom software development.
Looking for help with Salesforce development? Reach out to us at firstname.lastname@example.org. We are practitioners, not salespeople, so you’ll speak with an actual Credera consultant. We look forward to chatting to see how we might be able to help.