Configuration builder in NodeJS with TypeScript

In recent project that I am working on we had a requirement of creating a configuration builder that satisfies following requirements:

  • Ability to read default configuration from a file
  • Ability to read environment specific configuration from a file
  • Ability to override any defaults by command line options
  • Ability to override and/or access any defaults by the set as environment variables
  • Ability to update configuration by any module in the application without modifying default configuration module
  • Ability to read configuration from the database

Some background

The application in which this configuration builder needs to be developed follows a modular approach where each logically grouped functionalities are defined in a module and registers themselves to the main application. A similar approach to that of Angular 1.x modules. Each module receives application object while core app is going through module registration code, so the module can access any application properties that are public. The application is developed in NodeJS and the language of code that we are using TypeScript, so we had the ability to write classes, interfaces, and decorators to simplify code and make it more maintainable.

To put in simple words, we wanted a configuration module which can read arguments, environment variables, configuration files, and database while at the same time provide the ability to any other module in application to easily access and update configuration if required. Other modules will mostly add their configuration options into default config object, at application startup, so other places it can be accessed.

Making configuration builder

configuration_builder

To start with, I created default configuration class, which will do the default configuration read and hold the result in itself. This will then be attached to main Application object as the configuration of the application, every update to configuration will alter this class only.

export default class AppConfig implements IConfig {
    additionalConfigs: Types.Map;

    constructor() {
        this.additionalConfigs = {};
    }

    putExtra(key: string, value: any, override?: boolean): boolean {
        if (this.additionalConfigs[key] && !override) {
            return false;
        }
        this.additionalConfigs[key] = value;
        return true;
    }

    getExtra(key: string): any {
        return this.additionalConfigs[key];
    }
}

IConfig is an interface where I define how default configuration is going to look like. The additionalConfigs and related extra methods were added just in case someone decides not to populate default configuration space and want to add it in additional space then they can use it while updating configuration object.

Once default configuration structure is ready, I defined an interface called IConfigurationBuilder with a method called buildConfiguration that accepts an AppConfig object as an argument. The idea behind this is that if any module wants to contribute in configuration building activity then they can implement this interface, register itself to ConfiurationBuilder and rest will be handled by the builder to make sure that it’s buildConfiguration is executed while configuration was being built.

export interface IConfigBuilder {
    configure(config: AppConfig): Promise<any>;
}

After that, I defined a ConfigurationBuilder, which is responsible for building configuration on application startup by following all above requirements. It will first read default configuration and initialise AppConfig instance, then reads environment files and update configuration object, then read command line options and environment variables and add them to configuration object and finally go through different ConfigurationBuilders registered by different modules and call their buildConfiguration method which updates configuration object based on their requirement. Some default configuration builders we implemented are EnvAndOptConfigReader and DatabaseConfigReader.

export default class ConfigBuilder {
    private builders: Array<IConfigBuilder>;
    private config: AppConfig;

    constructor(configFilePath?: string) {
        this.builders = [];
        if (!configFilePath && fs.existsSync("config.json")) {
            configFilePath = path.join(process.cwd(), "config.json");
        }
        this.config = new AppConfig();
        const customConfig = require(configFilePath);
        _.merge(this.config, customConfig);
    }

    addConfigBuilder(builder: IConfigBuilder): ConfigBuilder {
        this.builders.push(builder);
        return this;
    }

    build(): Promise<any> {
        let defers: Array<Promise<any>> = [];
        this.builders.forEach((builder: IConfigBuilder) => {
            defers.push(builder.configure(this.config));
        });
        return Promise.all(defers).then(() => {
            return this.config;
        });
    }
}

As you can see in the constructor it reads default config.JSON file and initialise AppConfig with it. The `addConfigBuilder` method is supposed to be used by other modules to register their ConfigurationBuilders and finally `build` method loops over all registered builders and waits for them to finish updating configuration object by using deferred promise.

Advantages

Following are some advantages of this:

  • Clean and maintainable code
  • Easy to understand
  • Easy to extend – to add new configuration for a specific module is easy and does not require to update core code
  • Isolation

What else can be done?

In addition to this to make it easier, decorators can be added to the system which will auto-register your class to the builder. Same way decorators for other facilities can also be added like environment specific class loading. To do this you can either use some available auto-discovery plugin for typescript or implement a custom solution using metadata features of a decorator.

Do not reinvent wheel

Search for an available solution before implementing your own, there are some good solutions available online which can do similar stuff. The available solution at this point of time was not sufficient to cover all our application related structure and requirements which led to defining custom solution but for you, you might be able to find readily available solution in NPM repository.

Final note

Above code is just for illustration purpose and not production ready, you need to tweak it and add more validations/checks to make it more suitable for production.

Advertisements

Trying new tech stack for project – Good or bad?

If you search for ways to keep up to date with latest technologies, I’m sure you will find at least couple of articles stating “trying new technologies for each new project” as one of the ways to do it. As appealing and good this sounds to developers, it is not the best possible solution if you have a beginner team and you are selecting to replace the whole stack with a new one. Following are some of the reasons that I believe which can explain why selecting new technology stack that you have no experience with for new solution is not a good idea.

Firstly, the client is paying you to do develop a solution for him or her and not to help you learn something new. If you choose to try a completely new stack that you have no experience with for a new client project than it is obvious that you will spend some time in learning and then apply your knowledge in developing required solution. This could also go side by side as project continues but again no matter how fast you can learn something new, you are still making your client spend some bucks to help you learn.

In addition to this, getting a result that is most efficient or follows best practice all the way is not true when you are learning and doing things at the same time. You will be able to learn the new stack from a book or some online articles but that will only help in completing the job and not in doing the job well. For instance, you can easily find an online post explaining how to make an API call but as a developer, your job does not end there, you should be able to find a way to make it completely efficient. If you can not find the most efficient use of new technology stack then what’s the point of using it?

Lastly, it could be stressful to you and your team. As fun as it can be to learn and use new things, there is also a dark side where you have to find and solve issues in it which can be stressful sometimes. Because you have just started to use it, you mostly rely on your initial understanding and possibly online help to solve any defect that comes in your way. Sometimes, there could be an issue that no has ever experienced and it’s bugging since the last couple of days, everyone is pressuring your complete it, resulting in long hours and overtimes. In addition, if you have a team of a beginner then this could go worse. You will always be struggling with deadlines and at the end, client is not satisfied.

In summary, as listed above it might not be a good idea to try completely new stack for a new project; instead, you should gain some experience in it by doing some POC or side project and then go full force with next client project.

Does this mean one should never try something new in a project? No, this does not mean that you should not use any new technology in a new project, you can, of course, select to replace some part of your technology stack to with something better available out there. Replacing a part is does not hurt that much than replacing the whole stack.For instance, you can choose to move to NoSQL or BigData database instead of SQL database for next project based on your experience with previous projects.

If you have any thoughts or experience on same, please feel free to comment on this post to share it.

“Sell me this pen” – a developer’s take

Recently I came across an interesting question on Quora that is “What are best responses to ‘sell me this pen?’ question?“. After reading various answers from experts following is my interpretation: One should start by asking questions to the customer and slowly raising the need to buy a pen. Once you have these in place then next step would be to promote your pen and finally sell it. As soon as I got this summary in my mind, I started thinking to link same in software development world and following are my thoughts for same.

questions-1328466_640For the first point, that is about asking questions and raising the need for pen, as far as the raising need part is concerned, that is something client will take care as he/she wants to launch his product, that we are developing, in the market and increase awareness and need for same so everyone will use it and eventually buy it if it’s paid software. For the questioning part, I have been with developers that falls on both side of the equation that is one who asks bunch of questions to clarify things before starting implementation and one who directly jumps on coding without asking anything and raise doubts as they come which sometimes lead to a system which completely different from what customer envisioned.

Chiefly, new developers are falling in the later technique and one the reason that I believe could be the new Agile way of development. Because in Agile way, our development should be open to changes at any point in time, even late. One of the major drawbacks for such technique is new requirements that emerge as a result of some clarification and affecting scope or delivery of the product.

digital-1320699_640Continuing on the sell the pen thoughts, next step that I interpreted is to promote and sell. Again this mainly falls under client’s coat, but still there are some areas where developers and mostly designers can help. For instance, designing a brochure for a product that captures everyone eyes when distributed or doing SEO of web application to make sure it ends up in a higher position in search results. There is also a thing called supporting a software and I have seen freelancers and some companies as well who goes beyond expectations in this area and delight their customer by contributing to their success. After all, if the client is successful then that adds up to your success as well.

To conclude, I believe that in a completely different dimension if a software developer is being asked “sell me this pen” question than there are high chances that he/she will nail it. Let me know your thoughts or your interpretations and linking of “sell me this pen” question to IT industry.

%d bloggers like this: