The Art of Efficient Documentation: What to Document and What Not to Document

Documentation is a double-edged sword. On one hand, it provides essential guidance and clarity; on the other, it's a costly endeavor that can drain resources if not managed wisely. Understanding what to document and what to leave out is crucial, particularly in a competitive environment where efficiency is paramount.

What's important to note is that bad documentation is worse than no documentation.

Below are some thoughts on making strategic documentation decisions.

Why Documentation Matters

Documentation serves as the bridge between your product and its users, whether they are developers, end-users, or stakeholders. It can:

Enhance understanding of complex systems
Provide a historical record of decisions and changes
Facilitate onboarding and training

However, maintaining comprehensive documentation is expensive and time-consuming. Therefore, prioritizing what to document is essential.

What to Document

Core Concepts and Architecture

High-Level Architecture: Outline the system's overall structure and how its components interact. This helps new team members quickly grasp the big picture.
Core Algorithms and Data Structures: Describe any non-trivial algorithms or data structures that are central to the system's functionality.

API Documentation

Public APIs: Document all public-facing APIs thoroughly. Include endpoints, request/response formats, authentication methods, and error codes.
Internal APIs: Provide sufficient documentation for internal APIs to ensure seamless integration and maintenance.

Business Logic and Domain Models

Key Business Processes: Document the critical business logic that drives the application. Focus on workflows that are not immediately obvious from the code.
Domain Models: Describe the main entities and their relationships within the system. This helps in understanding the underlying business rules.

Deployment and Operational Procedures

Setup and Installation: Provide clear instructions for setting up the development environment, deploying the application, and configuring necessary services.
Monitoring and Troubleshooting: Document how to monitor the system's health and troubleshoot common issues.

Security and Compliance

Security Protocols: Outline the security measures in place, such as encryption, authentication, and authorization mechanisms.
Compliance Requirements: Document any industry-specific compliance requirements and how the system adheres to them.

What Not to Document

What's Already Documented

Avoid documenting things that are already documented. If your system is leveraging another system, then you should be able to lean on that documentation.

Obvious Code Behavior

Avoid documenting straightforward code that can be easily understood by reading it. For example, documenting a simple getter method is not only unnecessary, but it makes it more difficult for a skilled person to read.

Outdated Information

Don't document features or components that are deprecated or soon to be replaced. Focus on the current and future state of the system.

Excessive Detail

Skip overly detailed explanations of trivial implementations. Aim for clarity and conciseness, providing just enough information to understand the concept.

Where to Document

As we move towards a modern microservices approach, maintaining consistency and accessibility in documentation is key. Here's how to effectively manage your documentation:

Version Control Integration

Documentation in Repos: Store all documentation in the same repositories as the code, using markdown format. This ensures that the documentation is version-controlled and easily accessible.
Change Management: Capture changes directly in branches and tickets, documenting who requested the change, the design decision, and the resulting change. This creates instant portability and traceability for any developer.

Standardization

Project Standards: Each microservice project should have a standard for documentation. This includes the structure, format, and level of detail required.
Quality Assurance: Incorporate documentation updates into the release process. Ensure that the documentation is up-to-date, and that all relevant decisions are captured and traceable.

Tools and Formats

Markdown: Use markdown for all documentation to avoid creating a document swamp filled with outdated information. Markdown is easy to write, read, and maintain.
Flowcharting Tools: Utilize tools like Mermaid.js for creating flowcharts and diagrams. This provides the necessary visualization capabilities without complicating the documentation process and because your chart is in code, it is in version control.

How to Document

When it comes to documentation, using the right tools and formats is essential. Here's how to approach documenting different aspects of your system effectively:

Use Appropriate Languages and Formats

API Documentation: Use OpenAPI Specification to document APIs. This is a universally accepted standard that developers understand and can use directly. It also allows machines to interpret and test the API.
Settings and Configurations: Document settings within the application itself. There's no need to rewrite settings documentation elsewhere if the application already provides detailed descriptions.

Avoid Redundancy

Don't duplicate information that is already well-documented elsewhere. This only creates a maintenance burden and increases the risk of outdated information. Encourage team members to refer to existing resources such as user manuals, forums, and support groups.

Be Liberal with Sources

Provide ample references and sources of information within your documentation. This enables others to delve deeper into the subject matter, fostering a culture of continuous learning and discovery. By allowing team members to explore and learn more than they initially intended, you cultivate a more knowledgeable and self-sufficient team.

Remember, the highest value activity is increasing the productivity of the team.

Trust and Accountability

Trust your team to engage with the existing documentation. If individuals cannot be troubled to read and understand the provided resources, they should either trust those who have done the work or find a workplace with a larger budget for hand-holding. High-quality development demands a high-caliber team willing to put in the effort.

Building Effective Teams

Instead of pouring resources into exhaustive documentation, invest in building teams of good thinkers who can work effectively together. A strong team can:

Pair business problems with code solutions
Navigate and understand the codebase with minimal external documentation
Adapt to changes quickly without relying on outdated documents

What Not to Do: Cautionary Tales

Cautionary Tale #1

In a recent meeting, it was suggested that we capture all the information on how we handle a particular process, let's call it "X." However, "X" is managed by an off-the-shelf product that already comes with extensive documentation. What this person was really saying is they are too lazy to read the manual. This is a massive red flag and a cultural problem.

If team members cannot be troubled to do the work of reading the existing documentation, they become a time sink and a liability. Solving problems for people who lack the critical thinking skills of developing their own questions is a waste of time. They will continue to drain resources without adding value.

This person was essentially asking us to translate existing documentation into a format they could understand, despite the fact that all necessary information was readily available. The off-the-shelf system provided a comprehensive document store, a user group, and a forum for questions. Once you take the time to understand how things work, you can quickly access the application and see how everything is set up. Yes, this demands a higher caliber development team, but if you think good developers are expensive, try hiring the cheap ones.

Cautionary Tale #2

In another case, it was asked that we document the behavior of an API. For a trained developer, this request is beyond asinine. The OpenAPI Specification (OpenAPI Spec) is the preferred method for doing this. It's universally accepted and every good developer understands it. In fact, a quick Google search on "how to document an API" will likely turn up just this result. Eric Raymond's first tip in "How to Ask Questions the Smart Way" in the "Before You Ask" section is, "Try to find your answer by searching the web."

Again, we have a clear example of an individual who does not want to do the work, which should send off alarm signals. When you document your systems in an OpenAPI Spec, not only can developers use your spec, but machines can too. In subsequent releases, clients can immediately regression test your new version against their systems. Documenting correctly creates massive leverage.

Great coders can do things at impressive, sometimes unbelievable speed. A big reason for this is the tooling.

Conclusion

Documentation is essential, but it must be strategic. Focus on documenting the core concepts, business logic, APIs, and operational procedures that add value. Avoid documenting the obvious or trivial details.

Invest in writing clean, understandable code and building strong, collaborative teams. By doing so, you can maintain a lean, efficient approach to documentation that supports your competitive edge in the market.

Additionally, integrating documentation into version control systems ensures that it remains up-to-date, portable, and traceable, while markdown and flowcharting tools like Mermaid.js provide all the necessary capabilities without adding complexity.

Be liberal with your sources to foster a culture of continuous learning. Avoid catering to those unwilling to engage with existing resources, as this only perpetuates inefficiency. Use appropriate tools and formats to ensure your documentation is clear, concise, and beneficial for those who truly need it.