Compress Query Strings in JavaScript
Since query strings are limited depending on the browser, I wrote this function to grab redundant data variables and consolidate them into comma-separated values. For instance:
foo=1&foo=2&foo=3&blah=a&blah=b
would turn into…
foo=1,2,3&blah=a,b
This saves some room in the query string. Of course you need to write a trivial line to parse it on the server side, but this saves you some room for more variables in a query string. Here is my attempt at a JavaScript function that does this: (don’t use this, use John’s revised function below)
function compress(query) {
s = {type:[],value:[]}
$.each(data.split(‘&’), function(n){
parts = this.split(‘=’);
s.type[n] = parts[0];
s.value[n] = parts[1];
});
var csv = ‘’;
while(s.type.length > 0) {
value = s.value.shift();
type = s.type.shift();
while((pos = $.inArray(type, s.type)) >1) {1);
value += ‘,’ + s.value.shift();
s.type.shift();
}
csv += type + ‘=’ + value + ‘&’;
}
return csv.substring(0, csv.length
}
I messaged John Resig , and he wrote the function that was much smaller, and faster.
function compress(data){
var q = {}, ret = "";
data.replace(/([^=&]+)=([^&]*)/g, function(m, key, value){
q[key] = (q[key] ? q[key] + "," : "") + value;
});
for ( var key in q )
ret = (ret ? ret + "&" : "") + key + "=" + q[key];
return ret;
}
UPDATE: John Resig followed up with a post of his own, titled search and don’t replace .
How John’s Script Works
You may be asking, but how does this work? – let me explain. First, the replace function accepts a regular expression as the first parameter, and the second parameter can be a string or a function.
replace(/([^=&]+)=([^&]*)/g, function(m, key, value){
As far as I’m concerned, that lines is the magic of the script so I will go furthur.
/([^=&]+)=([^&]*)/g
This is the regular expression that matches data that starts with ampersand and goes until the end or until the next ampersand. This extracts the data. The /g part loops through each match instead of just one, and does a replace on it.
The function he is passing in has the three parameters set by the regexp."m"is the match,"key"is the unique identifier, and"value"is the entire string that the regexp is running against.
Next he sets the object based on the key, if it exists already than add a comma between after the current value followed by the next value.
q[key] = (q[key] ? q[key] +",":"") + value;
The last part of the code is simply looping through the, “q” object and outputting our new compressed query string. Brilliant.
9 comments
Good stuff Marc. A few clarifying points if that is ok:
m = the string that matches the pattern (for example, foo=1)
key = first subexpression match (the result of what is matched from the rules in the parenthesis. In the case of foo=1, this would be foo)
value = second subexpression match (the result of what is matched from the rules in the parenthesis. In the case of foo=1, this would be 1)
I ran out of space!
The three arguments are not set by the regexp, but rather, these arguments are defined by what the replacement argument (in this case, a function), can accept. The first argument it can accept is the matched pattern (m), the second, etc, arguments represent matched subexpressions (there could be up to 99 of these I believe).
Thanks for your clarification, Leveille. John explained more clearly in his post: http://ejohn.org/blog/search-and-dont-replace/
Sorry about the character limit. I’ll be upgrading my blog soon. This one was just hacked together quick in PHP. I’m not proud of it, but it serves its purpose.
How about using the replace function trick with some recursion?
regex = /([^
Looks like my idea was swallowed:
http://xczimi-tek.blogspot.com/2008/03/compress-query-strings-with-javascript.html
Again, my blog comments scripting sucks which is probably why I get lots of traffic but few comments. Maybe I should up the priority of reprogramming my blog.
I had the desire to further compress querystrings with a number of records such as:
viewIDs=6653,7258,14600,15104,6521,1436,2005,2006,2007,2008,2009The first thing i tried was to convert each number after the initial ID into a relative measure of distance from the first so you could get:
viewIDs=6653,605,-7342,504,8583,-5085,569,1,1,1,1Then you can use javascript base36 encoding Number(“605”.toString(36)) and die happy or take advantage of the fact that javascript base36 encoding is all-lowercase and further compress the redundant 1,1,1,1 sections using the remaining 26 uppercase characters—ie: representing -1 and +1 with “Q” and “P” respectively and come up with a final string:
viewIDs=6653,gt,5ny,e0,-6mf,-3x9,ft,PPPPhere is a quick example where ‘result’ is an array of the original IDs
It`s very impressive technique – “function in replace” – but very slowly.
For example, I maked faster then twice code :
I tried to re-write flickr “contact parser” early and was defeated. I used exactly technique by John, but simply “split then run array” it were faster.
Then I was come upon this post end benchmark your code. Unfortunately timing were as with flickr.
great post thank you I did also needed the inverse function:
Which I wrote with the same techniques: