Mongoose gotchas

I am using Mongoose since last few months and while querying MongoDB in Node.js and getting results. While doing that I came up with some cases that made me scratch my head for more than couple hours just to figure our it is little ‘gotcha’ (may be because I did not read through all the docs of Mongoose).

Listing my cases below so that if anyone ever comes to same mistake and tries to find for solution online, this can be helpful.

I have tried all following scenarios in 4.0.6 version of Mongoose with MongoDB 3.0.4.

1. When you query using a model and in result you see bunch of fields but when you try to access them you get undefined.

Ex.: (assuming you have user collections with name and email field in it.)

var UserSchema = new Schema({
 name: String
});

var User = mongoose.model('User', UserSchema);
User.find({}, function(err, users) {
 users.forEach(function(user) {
 console.log(user); // this will print something like {'name':'harsh', 'email': 'abc@abc.com'}
 console.log(user.email); // this will print 'undefined'
 });
});

Solution:
You need to add missing field entries in Schema definition, mongoose only binds fields that are in schema definition while creating objects.
Change above schema to following:


var UserSchema = new Schema({
 name: String,
 email: String
});

And now when you run same example above it should print email id of that particular record.

2. Auto conversion of field values in ‘find’ and ‘aggregate’ query.

Ex.:


var UserSchema = new Schema({
 name: String,
 cTime: Number
});

var User = mongoose.model('User', UserSchema);
User.find({
 cTime: '123456' // here we are passing value as string but it works
 // some how mongoose converts it before query may be
}, function(err, users) {
 console.log(users); // print users that match the cTime value
});

Now if you try same thing with aggregate function then it will not work:


User.aggregate({
 $match: { cTime : '1427807167' } // mongoose does not convert it to number this time
}, function(err, users) {
 console.log(users); // this will be empty, no match
});

In case of aggregate you have to take care of conversion by your own. Most likely case for this is getting time or other numeric values in req.query and then directly using it in $match query of aggregate. Instead you should first convert it to specific type of field and then use that in query.


User.aggregate({
 $match: { cTime : parseInt(req.query.time) } // assuming req.query.time has the value for cTime
}, function(err, users) {
 console.log(users); // now this will be not empty and prints the matching records.
});

3. Updating array items and calling save

When you just update an item in array of objects and call save on it, it will update the array in database. To make it work you need to mark the array field as modified.
user.markModified(‘array’); // this will mark array field of user object
if you call user.save after this then it will update array in database.

More info on this: http://stackoverflow.com/questions/17865081/updating-array-within-mongodb-record-with-mongoose

4. Listen to db connection drop event

This is probably not a gotcha, but it really made me work about an hour before knowing the solution. When connection to database is dropped for some reason then you will get it as ‘error’ event and not in ‘disconnected’ event. I tried to listed to ‘disconnected’ event and do reconnect but found that it does not even call it when db connection is crashed for some reason. It listens on ‘error’ event in that case so if you are planning to catch event for db connection failure and do operation based on it.

That’s all for now.

If you have seen any more while working with it, do let all know by commenting on the post. I will also update it with more as I come to know.

Samsung Galaxy Note II Keyboard space issue

As Samsung is launching new phones in its popular Note series, the old phone in the series is getting in trouble. Recently Note II owners got new system update which was approximately 104.88 MB. This update might have bring some fixes into system but left over an irritating issue with Samsung Keyboard.

Those who have used Samsung phones, especially Note Series, might have familiar with Predictive Text feature of keyboard. This feature actually predicts what user is going to type and provide some suggestions from which user can select and also tries to predict next word user would probably use thus saving users time and energy while typing. Another good functionality with keyboard was ‘Auto Spacing’, this feature is for automatically inserting spaces between words. This both  feature together makes it super easy for user to type something.

The recent update in Samsung Galaxy Note II has introduced a bug in Auto Spacing feature that is it inserts 2 spaces instead of 1. So if you type ‘he’ it will give you suggestions ‘hello’ and when you click on suggestion it will write ‘hello  ‘; 2 spaces after hello. So a simple sentence like ” Hi, How are you?’ becomes ‘Hi , How  are  you ? ‘ with predictive text and auto spacing on. If you disable auto spacing than it does not include any space at all, so its either 2 or nothing for space. This is very annoying for those who are spending their most of time messaging and doing social updates, and using note 2 phone.

Someone has also reported a bug into Samsung developer forum, but still there is no answer from Samsung; so user’s have to wait until Samsung sends a new update which resolves this. Meanwhile there are some alternative keyboards available as suggested by some user on forum.

So if you are using Samsung Galaxy Note II, installed latest updates and now getting unwanted spaces between words than You are not alone my friend, there are lots of others out there, including me (Yes I do own Samsung Galaxy Note II).

Some people have also reported to have similar issues with other Samsung phones like S4, so this might not be only for Note II.

If anyone of you surprisingly able to find a unique and perfect solution or get latest news about this, please comment on this post so others can know as well.

Reference: Samsung Developer Forum


 

Update (1 Jan 2015)

Finally got a new update from Samsung and it resolved this annoying issue, now everything works perfect as far as I can see.  :)

Simple Bootstrap CV

Want to build your online resume?

Harsh Raval  My Self   Bootstrap CV
Here is the chance to do same by using the provide template. It is a single page resume and is ready-to-go, just replace the content with yours and your online resume will be ready in just minutes. It is simple and based on the default bootstrap theme and its only html, css and little bit js.

The template is made with following things:

The whole source code is available on GitHub so you can easily grab it and use it. Don’t forget to star the GitHub repo if you like the template. For demo click the image.

Thanks🙂

Simple REST client in java

Today most of the mobile applications that used to communicate to some server use REST services. These services are also common practice to use with JavaScript or jQuery. Right now I know 2 ways to create client for REST service in java and in this article I will try to demonstrate both the ways I know hoping that it will help someone in some way.

1. Using Apache HttpClient

The Apache HttpClient library simplifies handling HTTP requests. To use this library you have to download the binaries with dependencies from their website.
Here is the code for HTTP GET method:

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
public class Test {
	public static void main(String[] args) throws ClientProtocolException, IOException {
		HttpClient client = new DefaultHttpClient();
		HttpGet request = new HttpGet("http://restUrl");
		HttpResponse response = client.execute(request);
		BufferedReader rd = new BufferedReader (new InputStreamReader(response.getEntity().getContent()));
		String line = "";
		while ((line = rd.readLine()) != null) {
		  System.out.println(line);
		}
	}
}

And for Post method; for sending simple string in post:

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.DefaultHttpClient;
public class Test {
	public static void main(String[] args) throws ClientProtocolException, IOException {
		HttpClient client = new DefaultHttpClient();
		HttpPost post = new HttpPost("http://restUrl");
		StringEntity input = new StringEntity("product");
		post.setEntity(input);
		HttpResponse response = client.execute(post);
		BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
		String line = "";
		while ((line = rd.readLine()) != null) {
			System.out.println(line);
		}
	}
}

You can also send full JSON or XML of a POJO by putting String representing JSON or XML as a parameter of StringEntity and then set the input content type. Something like this:

StringEntity input = new StringEntity("{\"name1\":\"value1\",\"name2\":\"value2\"}"); //here instead of JSON you can also have XML
input.setContentType("application/json");

For JSON you can use JSONObject to create string representation of JSON.

JSONObject json = new JSONObject();
json.put("name1", "value1");
json.put("name2", "value2");
StringEntity se = new StringEntity( json.toString());

And for sending multiple parameter in post request:

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.List;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.message.BasicNameValuePair;
public class Test {
	public static void main(String[] args) throws ClientProtocolException, IOException {
		HttpClient client = new DefaultHttpClient();
		HttpPost post = new HttpPost("http://restUrl");
		List nameValuePairs = new ArrayList(1);
		nameValuePairs.add(new BasicNameValuePair("name", "value")); //you can as many name value pair as you want in the list.
		post.setEntity(new UrlEncodedFormEntity(nameValuePairs));
		HttpResponse response = client.execute(post);
		BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
		String line = "";
		while ((line = rd.readLine()) != null) {
			System.out.println(line);
		}
	}
}

2. Using Jersey

Jersey is the reference implementation forJSR-311 specification, the specification of REST support in Java. Jersey contains basically a REST server and a REST client. it provides a library to communicate with the server producing REST services. For http get method:

import java.io.IOException;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.UriBuilder;
import org.apache.http.client.ClientProtocolException;
import com.sun.jersey.api.client.Client;
import com.sun.jersey.api.client.WebResource;
import com.sun.jersey.api.client.config.ClientConfig;
import com.sun.jersey.api.client.config.DefaultClientConfig;
public class Test {
	public static void main(String[] args) throws ClientProtocolException, IOException {
		ClientConfig config = new DefaultClientConfig();
		Client client = Client.create(config);
		WebResource service = client.resource(UriBuilder.fromUri("http://restUrl").build());
		// getting XML data
		System.out.println(service. path("restPath").path("resourcePath").accept(MediaType.APPLICATION_JSON).get(String.class));
		// getting JSON data
		System.out.println(service. path("restPath").path("resourcePath").accept(MediaType.APPLICATION_XML).get(String.class));
	}
}

There are also other media formats in which you can get the response like PLAIN or HTML.
And for HTTP POST method:

import java.io.IOException;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.MultivaluedMap;
import javax.ws.rs.core.UriBuilder;
import org.apache.http.client.ClientProtocolException;
import com.sun.jersey.api.client.Client;
import com.sun.jersey.api.client.ClientResponse;
import com.sun.jersey.api.client.WebResource;
import com.sun.jersey.api.client.config.ClientConfig;
import com.sun.jersey.api.client.config.DefaultClientConfig;
import com.sun.jersey.core.util.MultivaluedMapImpl;
public class Test {
	public static void main(String[] args) throws ClientProtocolException, IOException {
		ClientConfig config = new DefaultClientConfig();
		Client client = Client.create(config);
		WebResource webResource = client.resource(UriBuilder.fromUri("http://restUrl").build());
		MultivaluedMap formData = new MultivaluedMapImpl();
		formData.add("name1", "val1");
		formData.add("name2", "val2");
		ClientResponse response = webResource.type(MediaType.APPLICATION_FORM_URLENCODED_TYPE).post(ClientResponse.class, formData);
		System.out.println("Response " + response.getEntity(String.class));
	}
}

If you are using your POJO in the POST then you can do something like following:

ClientResponse response = webResource.path("restPath").path("resourcePath").
type(MediaType.APPLICATION_JSON).accept(MediaType.APPLICATION_JSON).post(ClientResponse.class, myPojo);
System.out.println("Response " + response.getEntity(String.class));

Here myPojo is an instance of custom POJO class.
You can also use Form class from Jersey to submit multiple parameters in POST request:

import java.io.IOException;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.UriBuilder;
import org.apache.http.client.ClientProtocolException;
import com.sun.jersey.api.client.Client;
import com.sun.jersey.api.client.ClientResponse;
import com.sun.jersey.api.client.WebResource;
import com.sun.jersey.api.client.config.ClientConfig;
import com.sun.jersey.api.client.config.DefaultClientConfig;
import com.sun.jersey.api.representation.Form;
public class Test {
	public static void main(String[] args) throws ClientProtocolException, IOException {
		ClientConfig config = new DefaultClientConfig();
		Client client = Client.create(config);
		WebResource service = client.resource(UriBuilder.fromUri("http://restUrl").build());
		Form form = new Form();
		form.add("name1", "value1");
		form.add("name2", "value1");
		ClientResponse response = service.path("restPath").path("resourcePath").
		type(MediaType.APPLICATION_FORM_URLENCODED).post(ClientResponse.class, form);
		System.out.println("Response " + response.getEntity(String.class));
	}
}

Let me know if you find any difficulty while using these code samples. Feel free to comment.🙂

Gateway interaction in Smack API

Gateway interaction is described by XEP-0100. The query xmlns (namespace) used for this is same as IN-Band registration, that is “jabber:iq:register”. Smack API provides an easy way handle gateway interaction with the server. It has predefined classes for the required namespace and operations which one can easily use and do the thing. In this article I will explain how one can register, edit, unregister and retrieve gateway information using Smack library. I have listed both XML and code using Smack API for the purpose. I have used Openfire server with Kraken plugin in it for gateway related operations described in this article.

1. Gateway service discovery

First you have to send an IQ-get with xmlns of the service discovery info to the Gateway. The XML would look like this:

<iq id="uLQX1-3" to="gtalk.mtHost" type="get">
	<query xmlns="http://jabber.org/protocol/disco#info"></query>
</iq>

And in return Gateway returns identity information like this:

<iq id="uLQX1-3" to="user1@myHost/Smack" from="gtalk.myHost" type="result">
	<query xmlns="http://jabber.org/protocol/disco#info">
		<identity category="gateway" name="Google Talk Transport" type="xmpp"/>
		<feature var="http://jabber.org/protocol/disco#info"/>
		<feature var="http://jabber.org/protocol/disco#items"/>
		<feature var="jabber:iq:gateway"/>
		<feature var="jabber:iq:register"/>
		<feature var="jabber:iq:version"/>
		<feature var="vcard-temp"/>
	</query>
</iq>

By this you can identify which namespace features are supported by the gateway. To register and/or unregister the gateway “jabber:iq:register” support is required. This response can differ at your side based on your server and Gateway support. Now the code using Smack API for this operation is as follows:

DiscoverInfo iq = new DiscoverInfo();
iq.setType(IQ.Type.GET);
iq.setTo("gtalk.myHost");
PacketCollector collector = connection.createPacketCollector(new PacketIDFilter(iq.getPacketID()));
connection.sendPacket(iq);
IQ response = (IQ)collector.nextResult(SmackConfiguration.getPacketReplyTimeout());
collector.cancel();
System.out.println(response.toXML());

And to check for specific feature is supported or not and getting information of items in response do the following:

if (response != null && !response.getType().equals(IQ.Type.ERROR) && response instanceof DiscoverInfo) {
	DiscoverInfo info = (DiscoverInfo) response;
	Iterator<Identity> identities = info.getIdentities();
	while (identities.hasNext()) {
		DiscoverInfo.Identity identity = (DiscoverInfo.Identity) identities.next();
		System.out.println("identity name -> " + identity.getName());
		System.out.println("identity category -> " + identity.getCategory());
		System.out.println("identity type -> " + identity.getType());
	}
	System.out.println(info.containsFeature("jabber:iq:register"));
}

2. Get information of gateway

To query gateway id about which information is required to register, you need to send following xml:

<iq id="VT12T-4" to="gtalk.myHost" type="get">
	<query xmlns="jabber:iq:register"></query>
</iq>

In response server will respond something like this:

<iq id="VT12T-4" to="user1@myHost/Smack" from="gtalk.myHost" type="result">
	<query xmlns="jabber:iq:register">
		<instructions>Please enter your e-mail address and password used with GMail and GTalk</instructions>
		<username></username>
		<password></password>
		<x xmlns="jabber:x:data" type="form">
			<instructions>Please enter your e-mail address and password used with GMail and GTalk.</instructions>
			<field label="Address" var="username" type="text-single"></field>
			<field label="Password" var="password" type="text-private"></field>
		</x>
		<x xmlns="jabber:iq:gateway:register"></x>
	</query>
</iq>

By this you will know that which fields are required for gateway registration. Here in this example 2 fields are require, one is username and another is password.
Code to do this in smack is:
1) Send GET request to gateway

Registration iq = new Registration();
iq.setType(IQ.Type.GET);
iq.setTo("gtalk.myHost");
connection.sendPacket(iq);

2) Get the response.

PacketCollector collector = connection.createPacketCollector(new PacketIDFilter(iq.getPacketID()));
IQ response = (IQ)collector.nextResult(SmackConfiguration.getPacketReplyTimeout());
collector.cancel();
System.out.println(response.toXML());

3) Process the response for extracting required information like this:

if (response != null && !response.getType().equals(IQ.Type.ERROR) && response instanceof Registration) {
Registration registration = (Registration) response;
System.out.println(registration.getInstructions());
	if (registration.getAttributes() != null && !registration.getAttributes().isEmpty()) {
		Map<String, String> map = registration.getAttributes();
		System.out.println(map);
	}
}

3. Register gateway:

To register a gateway using one has to send IQ SET with the required attributes:

<iq id="WiISc-2" to="gtalk.myHost" type="set">
	<query xmlns="jabber:iq:register">
		<username>user</username>
		<password>pass</password>
	</query>
</iq>

And as a successful response server will send back following xml:

<iq type='result' from='gtalk.myHost' to=user1@myHost/Smack' id='WiIsc-2'/>

And here is the code to do registration using Smack API:
1) Send an IQ set packet. For jabber:iq:register we will use Registration class from smack library:

Map<String, String> attributes = new HashMap<String, String>();
attributes.put("username", "user");
attributes.put("password", "pass");
Registration registration = new Registration();
registration.setType(IQ.Type.SET);
registration.setTo("aim.myHost");
registration.setAttributes(attributes);
connection.sendPacket(registration);

2) Get the response:

PacketCollector collector = connection.createPacketCollector(new PacketIDFilter(registration.getPacketID()));
IQ response = (IQ)collector.nextResult(SmackConfiguration.getPacketReplyTimeout());
collector.cancel();

4. Retrieve registered gateway information:

To retrieve registered gateway information one has to send an IQ GET packet with “jabber:iq:register” query to the gateway id (i.e. gtalk.myHost). This is similar what we did to get gateway information. In short when you can query gateway for information it will return the information with currently registered information, if any and if there there are no registered information then response is just about the required field names to register.

<iq id="VT12T-4" to="gtalk.myHost" type="get">
	<query xmlns="jabber:iq:register"></query>
</iq>

The gateway returns registration requirements with already registered information like following:

<iq type='result' id='VT12T-4'>
   <query xmlns='jabber:iq:register'>
      <registered/>
      <username>user</username>
      <password>pass</password>
   </query>
</iq>

One can do this thing in Smack API as follows:
1) Send IQ GET packet to gateway:

Registration message = new Registration();
message.setTo("gtalk.myHost");
message.setType(IQ.Type.GET);
connection.sendPacket(message);

2) Get and process the response returned by gateway:

PacketCollector collector = connection.createPacketCollector(new PacketIDFilter(message.getPacketID()));
IQ response = (IQ)collector.nextResult(SmackConfiguration.getPacketReplyTimeout());
collector.cancel();
if (response != null && !response.getType().equals(IQ.Type.ERROR) && response instanceof Registration) {
Registration registration = (Registration) response;
if (registration.getAttributes() != null && !registration.getAttributes().isEmpty()) {
Map<String, String> map = registration.getAttributes();
// map contains information about registration.
}
}

5. Edit registered information:

To edit registered information one has to first fetch information as shown in above section and then follow the same code for registering gateway. In simple words do point 3 & 4 form this article in reverse order.

6. Unregister gateway

To unregister a gateway first query for the registered gateway to see if any gateway information is registered or not. If it is registered then to unregister you just have to add a “remove” attribute in the IQ set as follows:

<iq type='set' from=user1@myHost/Smack' to='gtalk.myHost' id='VTiLk-4'>
   <query xmlns='jabber:iq:register'>
      <remove/> <!—remove is for unregisterting the gateway -->
   </query>
</iq>

This is very similar to the register gateway xml so difference in code is very small. Instead of adding username and password as attributes in Map you just add a “remove” attribute and everything else is same.

Remove these 2 lines from register gateway code:

attributes.put("username", "user");
attributes.put("password", "pass");

And add following line instead:

attributes.put("remove", ""); // no need to provide any value as remove tag is an empty tag

Everything else in the code is same. Response is also similar to the register gateway response.

This is it for now!!!🙂

Transparent JFrame using JNA

In Make JFrame transparent I had shown a way to making frame’s transparent using AWTUtilities class. But using that class results in access restriction compile time error, resolution in Eclipse is also shown in that post. Now here is the version using java natives. I have used Java Native Access (JNA) library to call native functions to get the things done.

What is Java Native Access (JNA)?

JNA provides Java programs easy access to native shared libraries (DLLs on Windows) without writing anything but Java code—no JNI or native code is required.JNA allows you to call directly into native functions using natural Java method invocation.

The Code

import javax.swing.JFrame;
import javax.swing.JSlider;
import javax.swing.event.ChangeEvent;
import javax.swing.event.ChangeListener;

import com.sun.jna.platform.WindowUtils;

public class TransparentFrame extends JFrame {
   public TransparentFrame() {
      setTitle("Transparent Frame");
      setSize(400,400);
      setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
      JSlider slider = new JSlider(JSlider.HORIZONTAL, 30, 100, 100);

      slider.addChangeListener(new ChangeListener() {
         @Override
         public void stateChanged(ChangeEvent e) {
            JSlider slider = (JSlider) e.getSource();
            if(!slider.getValueIsAdjusting()){
               WindowUtils.setWindowAlpha(TransparentFrame.this, slider.getValue()/100f);
            }
         }
      });
      add(slider);
      setVisible(true);
   }
   public static void main(String[] args) {
      SwingUtilities.invokeLater(new Runnable() {
	 @Override
	 public void run() {
	    new TransparentFrame();
	 }
      });
   }
}

Here WindowUtils class is provided in JNA jar (platform.jar). The method setWindowAlpha of WindowUtils class is used to make a window transparent. First argument of this method is your frame/window and second argument is alpha value. This class also has a method called setWindowTransparent, which can also be used to make window transparent .

Dependencies

You will need following 2 jars to run this program: (Both jar files are available to download on GitHub for JNA.)

  • jna.jar
  • platform.jar

To run above code on Windows, you will need to set “sun.java2d.noddraw” system property before calling the WindowUtils function.

System.setProperty("sun.java2d.noddraw", "true");

Output

Additional Notes

I have tested this code on following machines:

  • Windows XP service pack 3 (32 bit)
  • Windows 7 (32 bit)
  • Cent OS 5 (32 bit)

If you test it on other machines or have code for other machines using JNA for the same functionality then feel free to share it as comment to this post.