Advertisements

Configuration builder in NodeJS with TypeScript

In recent project that I am working on we had a requirement of creating a configuration builder that satisfies following requirements:

  • Ability to read default configuration from a file
  • Ability to read environment specific configuration from a file
  • Ability to override any defaults by command line options
  • Ability to override and/or access any defaults by the set as environment variables
  • Ability to update configuration by any module in the application without modifying default configuration module
  • Ability to read configuration from the database

Some background

The application in which this configuration builder needs to be developed follows a modular approach where each logically grouped functionalities are defined in a module and registers themselves to the main application. A similar approach to that of Angular 1.x modules. Each module receives application object while core app is going through module registration code, so the module can access any application properties that are public. The application is developed in NodeJS and the language of code that we are using TypeScript, so we had the ability to write classes, interfaces, and decorators to simplify code and make it more maintainable.

To put in simple words, we wanted a configuration module which can read arguments, environment variables, configuration files, and database while at the same time provide the ability to any other module in application to easily access and update configuration if required. Other modules will mostly add their configuration options into default config object, at application startup, so other places it can be accessed.

Making configuration builder

configuration_builder

To start with, I created default configuration class, which will do the default configuration read and hold the result in itself. This will then be attached to main Application object as the configuration of the application, every update to configuration will alter this class only.

export default class AppConfig implements IConfig {
    additionalConfigs: Types.Map;

    constructor() {
        this.additionalConfigs = {};
    }

    putExtra(key: string, value: any, override?: boolean): boolean {
        if (this.additionalConfigs[key] && !override) {
            return false;
        }
        this.additionalConfigs[key] = value;
        return true;
    }

    getExtra(key: string): any {
        return this.additionalConfigs[key];
    }
}

IConfig is an interface where I define how default configuration is going to look like. The additionalConfigs and related extra methods were added just in case someone decides not to populate default configuration space and want to add it in additional space then they can use it while updating configuration object.

Once default configuration structure is ready, I defined an interface called IConfigurationBuilder with a method called buildConfiguration that accepts an AppConfig object as an argument. The idea behind this is that if any module wants to contribute in configuration building activity then they can implement this interface, register itself to ConfiurationBuilder and rest will be handled by the builder to make sure that it’s buildConfiguration is executed while configuration was being built.

export interface IConfigBuilder {
    configure(config: AppConfig): Promise<any>;
}

After that, I defined a ConfigurationBuilder, which is responsible for building configuration on application startup by following all above requirements. It will first read default configuration and initialise AppConfig instance, then reads environment files and update configuration object, then read command line options and environment variables and add them to configuration object and finally go through different ConfigurationBuilders registered by different modules and call their buildConfiguration method which updates configuration object based on their requirement. Some default configuration builders we implemented are EnvAndOptConfigReader and DatabaseConfigReader.

export default class ConfigBuilder {
    private builders: Array<IConfigBuilder>;
    private config: AppConfig;

    constructor(configFilePath?: string) {
        this.builders = [];
        if (!configFilePath && fs.existsSync("config.json")) {
            configFilePath = path.join(process.cwd(), "config.json");
        }
        this.config = new AppConfig();
        const customConfig = require(configFilePath);
        _.merge(this.config, customConfig);
    }

    addConfigBuilder(builder: IConfigBuilder): ConfigBuilder {
        this.builders.push(builder);
        return this;
    }

    build(): Promise<any> {
        let defers: Array<Promise<any>> = [];
        this.builders.forEach((builder: IConfigBuilder) => {
            defers.push(builder.configure(this.config));
        });
        return Promise.all(defers).then(() => {
            return this.config;
        });
    }
}

As you can see in the constructor it reads default config.JSON file and initialise AppConfig with it. The `addConfigBuilder` method is supposed to be used by other modules to register their ConfigurationBuilders and finally `build` method loops over all registered builders and waits for them to finish updating configuration object by using deferred promise.

Advantages

Following are some advantages of this:

  • Clean and maintainable code
  • Easy to understand
  • Easy to extend – to add new configuration for a specific module is easy and does not require to update core code
  • Isolation

What else can be done?

In addition to this to make it easier, decorators can be added to the system which will auto-register your class to the builder. Same way decorators for other facilities can also be added like environment specific class loading. To do this you can either use some available auto-discovery plugin for typescript or implement a custom solution using metadata features of a decorator.

Do not reinvent wheel

Search for an available solution before implementing your own, there are some good solutions available online which can do similar stuff. The available solution at this point of time was not sufficient to cover all our application related structure and requirements which led to defining custom solution but for you, you might be able to find readily available solution in NPM repository.

Final note

Above code is just for illustration purpose and not production ready, you need to tweak it and add more validations/checks to make it more suitable for production.

Advertisements

Trying new tech stack for project – Good or bad?

If you search for ways to keep up to date with latest technologies, I’m sure you will find at least couple of articles stating “trying new technologies for each new project” as one of the ways to do it. As appealing and good this sounds to developers, it is not the best possible solution if you have a beginner team and you are selecting to replace the whole stack with a new one. Following are some of the reasons that I believe which can explain why selecting new technology stack that you have no experience with for new solution is not a good idea.

Firstly, the client is paying you to do develop a solution for him or her and not to help you learn something new. If you choose to try a completely new stack that you have no experience with for a new client project than it is obvious that you will spend some time in learning and then apply your knowledge in developing required solution. This could also go side by side as project continues but again no matter how fast you can learn something new, you are still making your client spend some bucks to help you learn.

In addition to this, getting a result that is most efficient or follows best practice all the way is not true when you are learning and doing things at the same time. You will be able to learn the new stack from a book or some online articles but that will only help in completing the job and not in doing the job well. For instance, you can easily find an online post explaining how to make an API call but as a developer, your job does not end there, you should be able to find a way to make it completely efficient. If you can not find the most efficient use of new technology stack then what’s the point of using it?

Lastly, it could be stressful to you and your team. As fun as it can be to learn and use new things, there is also a dark side where you have to find and solve issues in it which can be stressful sometimes. Because you have just started to use it, you mostly rely on your initial understanding and possibly online help to solve any defect that comes in your way. Sometimes, there could be an issue that no has ever experienced and it’s bugging since the last couple of days, everyone is pressuring your complete it, resulting in long hours and overtimes. In addition, if you have a team of a beginner then this could go worse. You will always be struggling with deadlines and at the end, client is not satisfied.

In summary, as listed above it might not be a good idea to try completely new stack for a new project; instead, you should gain some experience in it by doing some POC or side project and then go full force with next client project.

Does this mean one should never try something new in a project? No, this does not mean that you should not use any new technology in a new project, you can, of course, select to replace some part of your technology stack to with something better available out there. Replacing a part is does not hurt that much than replacing the whole stack.For instance, you can choose to move to NoSQL or BigData database instead of SQL database for next project based on your experience with previous projects.

If you have any thoughts or experience on same, please feel free to comment on this post to share it.

“Sell me this pen” – a developer’s take

Recently I came across an interesting question on Quora that is “What are best responses to ‘sell me this pen?’ question?“. After reading various answers from experts following is my interpretation: One should start by asking questions to the customer and slowly raising the need to buy a pen. Once you have these in place then next step would be to promote your pen and finally sell it. As soon as I got this summary in my mind, I started thinking to link same in software development world and following are my thoughts for same.

questions-1328466_640For the first point, that is about asking questions and raising the need for pen, as far as the raising need part is concerned, that is something client will take care as he/she wants to launch his product, that we are developing, in the market and increase awareness and need for same so everyone will use it and eventually buy it if it’s paid software. For the questioning part, I have been with developers that falls on both side of the equation that is one who asks bunch of questions to clarify things before starting implementation and one who directly jumps on coding without asking anything and raise doubts as they come which sometimes lead to a system which completely different from what customer envisioned.

Chiefly, new developers are falling in the later technique and one the reason that I believe could be the new Agile way of development. Because in Agile way, our development should be open to changes at any point in time, even late. One of the major drawbacks for such technique is new requirements that emerge as a result of some clarification and affecting scope or delivery of the product.

digital-1320699_640Continuing on the sell the pen thoughts, next step that I interpreted is to promote and sell. Again this mainly falls under client’s coat, but still there are some areas where developers and mostly designers can help. For instance, designing a brochure for a product that captures everyone eyes when distributed or doing SEO of web application to make sure it ends up in a higher position in search results. There is also a thing called supporting a software and I have seen freelancers and some companies as well who goes beyond expectations in this area and delight their customer by contributing to their success. After all, if the client is successful then that adds up to your success as well.

To conclude, I believe that in a completely different dimension if a software developer is being asked “sell me this pen” question than there are high chances that he/she will nail it. Let me know your thoughts or your interpretations and linking of “sell me this pen” question to IT industry.

Mongoose gotchas

I am using Mongoose since last few months and while querying MongoDB in Node.js and getting results. While doing that I came up with some cases that made me scratch my head for more than couple hours just to figure our it is little ‘gotcha’ (may be because I did not read through all the docs of Mongoose).

Listing my cases below so that if anyone ever comes to same mistake and tries to find for solution online, this can be helpful.

I have tried all following scenarios in 4.0.6 version of Mongoose with MongoDB 3.0.4.

1. When you query using a model and in result you see bunch of fields but when you try to access them you get undefined.

Ex.: (assuming you have user collections with name and email field in it.)

var UserSchema = new Schema({
 name: String
});

var User = mongoose.model('User', UserSchema);
User.find({}, function(err, users) {
 users.forEach(function(user) {
 console.log(user); // this will print something like {'name':'harsh', 'email': 'abc@abc.com'}
 console.log(user.email); // this will print 'undefined'
 });
});

Solution:
You need to add missing field entries in Schema definition, mongoose only binds fields that are in schema definition while creating objects.
Change above schema to following:


var UserSchema = new Schema({
 name: String,
 email: String
});

And now when you run same example above it should print email id of that particular record.

2. Auto conversion of field values in ‘find’ and ‘aggregate’ query.

Ex.:


var UserSchema = new Schema({
 name: String,
 cTime: Number
});

var User = mongoose.model('User', UserSchema);
User.find({
 cTime: '123456' // here we are passing value as string but it works
 // some how mongoose converts it before query may be
}, function(err, users) {
 console.log(users); // print users that match the cTime value
});

Now if you try same thing with aggregate function then it will not work:


User.aggregate({
 $match: { cTime : '1427807167' } // mongoose does not convert it to number this time
}, function(err, users) {
 console.log(users); // this will be empty, no match
});

In case of aggregate you have to take care of conversion by your own. Most likely case for this is getting time or other numeric values in req.query and then directly using it in $match query of aggregate. Instead you should first convert it to specific type of field and then use that in query.


User.aggregate({
 $match: { cTime : parseInt(req.query.time) } // assuming req.query.time has the value for cTime
}, function(err, users) {
 console.log(users); // now this will be not empty and prints the matching records.
});

3. Updating array items and calling save

When you just update an item in array of objects and call save on it, it will update the array in database. To make it work you need to mark the array field as modified.
user.markModified(‘array’); // this will mark array field of user object
if you call user.save after this then it will update array in database.

More info on this: http://stackoverflow.com/questions/17865081/updating-array-within-mongodb-record-with-mongoose

4. Listen to db connection drop event

This is probably not a gotcha, but it really made me work about an hour before knowing the solution. When connection to database is dropped for some reason then you will get it as ‘error’ event and not in ‘disconnected’ event. I tried to listed to ‘disconnected’ event and do reconnect but found that it does not even call it when db connection is crashed for some reason. It listens on ‘error’ event in that case so if you are planning to catch event for db connection failure and do operation based on it.

That’s all for now.

If you have seen any more while working with it, do let all know by commenting on the post. I will also update it with more as I come to know.

Samsung Galaxy Note II Keyboard space issue

As Samsung is launching new phones in its popular Note series, the old phone in the series is getting in trouble. Recently Note II owners got new system update which was approximately 104.88 MB. This update might have bring some fixes into system but left over an irritating issue with Samsung Keyboard.

Those who have used Samsung phones, especially Note Series, might have familiar with Predictive Text feature of keyboard. This feature actually predicts what user is going to type and provide some suggestions from which user can select and also tries to predict next word user would probably use thus saving users time and energy while typing. Another good functionality with keyboard was ‘Auto Spacing’, this feature is for automatically inserting spaces between words. This both  feature together makes it super easy for user to type something.

The recent update in Samsung Galaxy Note II has introduced a bug in Auto Spacing feature that is it inserts 2 spaces instead of 1. So if you type ‘he’ it will give you suggestions ‘hello’ and when you click on suggestion it will write ‘hello  ‘; 2 spaces after hello. So a simple sentence like ” Hi, How are you?’ becomes ‘Hi , How  are  you ? ‘ with predictive text and auto spacing on. If you disable auto spacing than it does not include any space at all, so its either 2 or nothing for space. This is very annoying for those who are spending their most of time messaging and doing social updates, and using note 2 phone.

Someone has also reported a bug into Samsung developer forum, but still there is no answer from Samsung; so user’s have to wait until Samsung sends a new update which resolves this. Meanwhile there are some alternative keyboards available as suggested by some user on forum.

So if you are using Samsung Galaxy Note II, installed latest updates and now getting unwanted spaces between words than You are not alone my friend, there are lots of others out there, including me (Yes I do own Samsung Galaxy Note II).

Some people have also reported to have similar issues with other Samsung phones like S4, so this might not be only for Note II.

If anyone of you surprisingly able to find a unique and perfect solution or get latest news about this, please comment on this post so others can know as well.

Reference: Samsung Developer Forum


 

Update (1 Jan 2015)

Finally got a new update from Samsung and it resolved this annoying issue, now everything works perfect as far as I can see.  🙂

Simple Bootstrap CV

Want to build your online resume?

Harsh Raval  My Self   Bootstrap CV
Here is the chance to do same by using the provide template. It is a single page resume and is ready-to-go, just replace the content with yours and your online resume will be ready in just minutes. It is simple and based on the default bootstrap theme and its only html, css and little bit js.

The template is made with following things:

The whole source code is available on GitHub so you can easily grab it and use it. Don’t forget to star the GitHub repo if you like the template. For demo click the image.

Thanks 🙂

2012 in review

The WordPress.com stats helper monkeys prepared a 2012 annual report for this blog.

Here’s an excerpt:

4,329 films were submitted to the 2012 Cannes Film Festival. This blog had 35,000 views in 2012. If each view were a film, this blog would power 8 Film Festivals

Click here to see the complete report.

%d bloggers like this: