Demystifying Code Generation: Building Programs That Build Programs
July 17, 2023

Keith Casey

Code generation is the art of writing programs that write other programs. The most common place to use code generation is for generating libraries. In those scenarios, you have a fixed specification in a domain-specific language (DSL) and a code generator uses that to create the function or API calls representing individual requests or pieces of functionality. While code generation seems simple at first, there are many sharp corners and hidden surprises in anything beyond the most trivial scenarios.

But before you decide whether to build a code generator or not, exploring the depths, complexity, and tradeoffs is key. Sometimes, this exploration will lead you to realize your time is better spent extending common tools to meet your unique use case. But there are many scenarios where creating a custom code generator is your best option. Before you explore, let’s consider one concrete scenario were you’d want a code generator and the best practices to make it useful in the long term.

When to Write a Custom Code Generator

Imagine creating a web service that requires email address validation. To ensure the email addresses are valid, the "correct" validation regular expression is wildly complex and obtuse. That said, there are a handful of easier aspects to check — like too many characters or missing an "@" — so we can start there. Initially, you rely on standard library methods like fmt.Errorf to report errors but will find that it doesn’t give the user enough information about the error. To counter that, you add two unique error codes, "email too long" and "invalid email," to make it easier for the user to understand why their email address won’t validate.

You could embed the unique error codes as ad hoc strings within existing strings, but this lacks structure and depends on downstream developers or the user to understand your new pseudo-convention. Alternatively, you could create a function that takes a unique error code as a parameter but that still relies on strings and will lead to inconsistent implementations. So how can you uplevel the implementation?

One solution is to generate unique functions for each error you want to handle. This allows for specialized error types, facilitating observability and enabling higher-level code components to work with specific error conditions. However, manually defining all these functions is time-consuming, especially for programs with hundreds or thousands of errors.

This is where writing a custom code generator comes in handy. By defining errors in a reusable format like a YAML file, you can generate the necessary functions. This approach simplifies error management, promotes consistency, reuses these checks across projects and enables the generation of functions for multiple programming languages. You can create a unified error repository for the software, ensuring clear communication and streamlined error handling.

Code Generation Best Practices

When writing a custom code generator, you should first consider the scope and complexity of creating it. There are always trade-offs between customization and simplicity, and developers should aim to understand these up front before diving in. Once you decide to write a custom code generator, developers must keep these best practices in mind.

1. Use comments to prevent edits: You should use comments to give other developers instructions on how to use and edit the code. Comments can tell other teammates not to modify the generated code directly and point them to the source of truth where they should make changes.

2. Isolate generated files: Developers should separate generated files from human-written code by using distinct suffixes or separate directories. Isolation enhances developer experience and makes it easier to identify machine-generated files, simplifying tasks like file filtering or removing outdated generated files.

3. Use a consistent template structure: A consistent template file is crucial for writing an effective code generator because it promotes a standardized format, reduces duplication, and allows for easy modification and maintenance. It facilitates collaboration among developers and enables extensibility for future enhancements.

How to Leverage Code Generation

If you’re considering code generation, start small. Write custom tooling that you never plan on releasing to the world. Explore, play, break things, figure out strengths and weaknesses and find the bounds of what’s possible. That way, when you inevitably consider code generation in practice, you’ll better understand the effort it takes to write and maintain a custom code generator or tailor something that already exists.

With the right tools and practices, code generation is an accessible and valuable tool in your toolbox. It offers a straightforward approach to automating repetitive tasks and establishing conventions. It’s one of the many ways you can solve complex problems, and it just might help you do it in fewer lines of code.

Keith Casey is Director of Product Marketing at ngrok
Share this

Industry News

December 06, 2023

ngrok unveiled its JavaScript and Python SDKs, enabling developers to programmatically serve their applications and manage traffic by embedding ingress with a single line of code.

December 06, 2023

Data Theorem introduced API Attack Path Visualization capabilities for the protection of APIs and the software supply chain.

December 05, 2023

Security Journey announced support for WCAG, SCIM and continued compliance with SOC2 Type 2, which are leading industry standards.

December 05, 2023

Vercel announced a new suite of features for its Developer Experience (DX) Platform, made for enterprise teams with large codebases.

December 04, 2023

Atlassian Corporation has completed the acquisition of Loom, a video messaging platform that helps users communicate through instantly shareable videos.

December 04, 2023

Orca Security announced that the Orca Cloud Security Platform has achieved the Amazon Web Services (AWS) Built-in Competency.

November 30, 2023

Parasoft, a global leader in automated software testing solutions, today announced complete support for MISRA C++ 2023 with the upcoming release of Parasoft C/C++test 2023.2.

November 30, 2023 achieved the Amazon Elastic Kubernetes Service (Amazon EKS) Ready designation from Amazon Web Services (AWS).

November 29, 2023

CircleCI implemented a gen2 GPU resource class, leveraging Amazon Elastic Compute Cloud (Amazon EC2) G5 instances, offering the latest generation of NVIDIA GPUs and new images tailored for artificial intelligence/machine learning (AI/ML) workflows.

November 29, 2023

XM Cyber announced new capabilities that provide complete and continuous visibility into risks and vulnerabilities in Kubernetes environments.

November 29, 2023

PerfectScale has achieved the Amazon Elastic Kubernetes Service (Amazon EKS) Ready designation from Amazon Web Services (AWS).

November 28, 2023

BMC announced two new product innovations, BMC AMI DevX Code Insights and BMC AMI zAdviser Enterprise.

November 28, 2023

Rafay Systems announced the availability of the Rafay Cloud Automation Platform — the evolution of its Kubernetes Operations Platform — to enable platform teams to deliver automation and self-service capabilities to developers, data scientists and other cloud users.

November 28, 2023

Bitrise is integrating with Amazon Web Services (AWS) to provide compliance-conscious companies with greater access to CI/CD capabilities for mobile app development.

November 28, 2023

Armory announced a new unified declarative deployment capability for AWS Lambda.