Major bug with BlackBerry browser and multiple cookies

This is a long post.

Having investigated this issue for the last few days, I believe that there is a significant issue with the cookie implementation in BlackBerry browsers using the default Internet Browser. I haven’t been able to test the recently-released BlackBerry Bold (9000), which is bundled with the new BlackBerry Browser, but as far as I can determine, devices like the 8800 and 8820 are affected.

The problem occurs when a site attempts to set multiple cookies. Although they are stored on the device, the cookies are returned to the server haphazardly. Sometimes all cookies are returned; more often it’s only the first or last cookie that was set that is passed back.

The following is a TCP dump of traffic to a particular server. I’ve bolded the relevant parts [update: well, I did at one time] and changed a few names. The server is not load-balanced, and the headers are not altered on my end in any way.

GET / HTTP/1.0
profile: http://www.blackberry.net/go/mobile/profiles/uaprof/8820/4.2.2.rdf
x-wap-profile: "http://www.blackberry.net/go/mobile/profiles/uaprof/8820/4.2.2.rdf"
Accept: application/vnd.rim.html,text/html,application/xhtml+xml,application/vnd.wap.xhtml+xml,text/vnd.sun.j2me.app-descriptor,image/vnd.rim.png,image/jpeg,application/x-vnd.rim.pme.b,image/gif;anim=1,application/vnd.rim.jscriptc;v=0-8-8,application/x-javascript,application/vnd.rim.css;v=1,text/css;media=handheld,application/vnd.wap.wmlc;q=0.9,application/vnd.wap.wmlscriptc;q=0.7,text/vnd.wap.wml;q=0.7,*/*;q=0.5
Accept-Charset: ISO-8859-1,UTF-8,US-ASCII,UTF-16BE,windows-1252,UTF-16LE,windows-1254,KOI8-R,windows-1251,windows-1255,windows-1256,windows-1250
Accept-Language: en-US,en;q=0.5
User-Agent: BlackBerry8820/4.2.2 Profile/MIDP-2.0 Configuration/CLDC-1.1 VendorID/100
Host: cookie.example.com
Via: BISB_3.3.0.45, 1.1 pmds76.bisb1.blackberry:3128 (squid/2.5.STABLE12)
X-Forwarded-For: unknown
Cache-Control: max-age=259200
Connection: keep-alive

HTTP/1.1 200 OK
Date: Wed, 27 Aug 2008 15:56:19 GMT
Set-Cookie: cookie1=o4s9o298mbqjf5qnjmgad76rs0; path=/; domain=cookie.example.com
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: cookie2=en_us; expires=Thu, 27-Aug-2009 15:56:19 GMT; path=/; domain=cookie.example.com
Content-Language: en_us
Set-Cookie: cookie3=2d14a216211bc7f1a1b03fa747d1087e; expires=Mon, 01-Sep-2008 06:59:59 GMT; path=/; domain=cookie.example.com
Vary: Accept-Encoding
Content-Length: 7975
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=utf-8

Here we set three cookies: cookie1, cookie2, and cookie3.

GET /subsequent/request HTTP/1.0
User-Agent: BlackBerry8820/4.2.2 Profile/MIDP-2.0 Configuration/CLDC-1.1 VendorID/100
profile: http://www.blackberry.net/go/mobile/profiles/uaprof/8820/4.2.2.rdf
Referer: http://cookie.example.com/
Host: cookie.example.com
Cookie: cookie1=o4s9o298mbqjf5qnjmgad76rs0
Via: 1.1 pmds76.bisb1.blackberry:3128 (squid/2.5.STABLE12)
X-Forwarded-For: unknown
Cache-Control: max-age=259200
Connection: keep-alive

HTTP/1.1 200 OK
Date: Wed, 27 Aug 2008 15:56:21 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: cookie2=en_us; expires=Thu, 27-Aug-2009 15:56:21 GMT; path=/; domain=cookie.example.com
Content-Language: en_us
Set-Cookie: cookie3=a082e222cde7b4aedb6a8b42f2723849; expires=Mon, 01-Sep-2008 06:59:59 GMT; path=/; domain=cookie.example.com
Vary: Accept-Encoding
Content-Length: 3701
Keep-Alive: timeout=5, max=99
Connection: Keep-Alive
Content-Type: text/css

The BlackBerry (an 8820) returns only one cookie back (cookie1), so the server tries to resend the cookies it did not receive. Keep in mind the expiration times here were set in the future for this request.

GET / HTTP/1.0
User-Agent: BlackBerry8820/4.2.2 Profile/MIDP-2.0 Configuration/CLDC-1.1 VendorID/100
profile: http://www.blackberry.net/go/mobile/profiles/uaprof/8820/4.2.2.rdf
Referer: http://cookie.example.com/
Host: cookie.example.com
Cookie: cookie1=o4s9o298mbqjf5qnjmgad76rs0
Via: 1.1 pmds76.bisb1.blackberry:3128 (squid/2.5.STABLE12)
X-Forwarded-For: unknown
Cache-Control: max-age=259200
Connection: keep-alive

HTTP/1.1 200 OK
Date: Wed, 27 Aug 2008 15:56:21 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: cookie2=en_us; expires=Thu, 27-Aug-2009 15:56:22 GMT; path=/; domain=cookie.example.com
Content-Language: en_us
Set-Cookie: cookie3=270773eec992a03fd63a7b74b2f5ef9a; expires=Mon, 01-Sep-2008 06:59:59 GMT; path=/; domain=cookie.example.com
Vary: Accept-Encoding
Connection: close
Content-Type: text/html; charset=utf-8

This is the same as above. The next GET is really interesting:

GET / HTTP/1.0
User-Agent: BlackBerry8820/4.2.2 Profile/MIDP-2.0 Configuration/CLDC-1.1 VendorID/100
profile: http://www.blackberry.net/go/mobile/profiles/uaprof/8820/4.2.2.rdf
Referer: http://cookie.example.com/
Host: cookie.example.com
Cookie: cookie2=en_us
Cookie: expires=Thu, 27-Aug-2009 15:57:03 GMT
Via: 1.1 pmds76.bisb1.blackberry:3128 (squid/2.5.STABLE12)
X-Forwarded-For: unknown
Cache-Control: no-cache, max-age=259200
Connection: keep-alive

HTTP/1.1 200 OK
Date: Wed, 27 Aug 2008 15:57:05 GMT
Set-Cookie: cookie1=32mrdio18lvpg6cduhnt1prmj5; path=/; domain=cookie.example.com
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: cookie2=en_us; expires=Thu, 27-Aug-2009 15:57:05 GMT; path=/; domain=cookie.example.com
Content-Language: en_us
Set-Cookie: cookie3=2d0b160a97e20469aad34e01890adbfb; expires=Mon, 01-Sep-2008 06:59:59 GMT; path=/; domain=cookie.example.com
Vary: Accept-Encoding
Connection: close
Content-Type: text/html; charset=utf-8

The BlackBerry sends back cookie2 and “expires” in two headers, as if they are two separate cookies!

For comparison, here’s a similar request from the same device accessing the same URL, but using Opera Mini instead. It performs as expected.

GET / HTTP/1.1
User-Agent: Opera/9.50 (J2ME/MIDP; Opera Mini/4.1.11355/546; U; en)
Host: cookie.example.com
Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1
Accept-Language: en
Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0
Connection: Keep-Alive
X-OperaMini-Features: advanced, file_system, folding
X-OperaMini-Phone-UA: BlackBerry
X-OperaMini-Phone: RIM #
x-forwarded-for: 206.53.144.82, unknown

HTTP/1.1 200 OK
Date: Wed, 27 Aug 2008 17:14:47 GMT
Set-Cookie: cookie1=dpr07co4i5dchtgmk8g3gdt0b6; path=/; domain=cookie.example.com
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: cookie2=en_us; expires=Thu, 27-Aug-2009 17:14:47 GMT; path=/; domain=cookie.example.com
Content-Language: en_us
Set-Cookie: cookie3=1ce651a62bfe62a25faff44e5e5d500b; expires=Mon, 01-Sep-2008 06:59:59 GMT; path=/; domain=cookie.example.com
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 2176
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=utf-8

GET /image/example.gif HTTP/1.1
User-Agent: Opera/9.50 (J2ME/MIDP; Opera Mini/4.1.11355/546; U; en)
Host: cookie.example.com
Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1
Accept-Language: en
Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0
Referer: http://cookie.example.com/
Cookie: cookie1=dpr07co4i5dchtgmk8g3gdt0b6; cookie3=1ce651a62bfe62a25faff44e5e5d500b; cookie2=en_us
Cookie2: $Version=1
Connection: Keep-Alive, TE
TE: deflate, gzip, chunked, identity, trailers
X-OperaMini-Features: advanced, file_system, folding
X-OperaMini-Phone-UA: BlackBerry
X-OperaMini-Phone: RIM #
x-forwarded-for: 206.53.144.82, unknown

HTTP/1.1 200 OK
Date: Wed, 27 Aug 2008 17:14:49 GMT
Last-Modified: Wed, 20 Aug 2008 19:36:32 GMT
ETag: "2c"
Accept-Ranges: bytes
Content-Length: 44
Keep-Alive: timeout=5, max=99
Connection: Keep-Alive
Content-Type: image/gif

GET / HTTP/1.1
User-Agent: Opera/9.50 (J2ME/MIDP; Opera Mini/4.1.11355/546; U; en)
Host: cookie.example.com
Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1
Accept-Language: en
Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0
Cookie: cookie1=dpr07co4i5dchtgmk8g3gdt0b6; cookie3=1ce651a62bfe62a25faff44e5e5d500b; cookie2=en_us
Cookie2: $Version=1
Cache-Control: no-cache
Connection: Keep-Alive, TE
TE: deflate, gzip, chunked, identity, trailers
X-OperaMini-Features: advanced, file_system, folding
X-OperaMini-Phone-UA: BlackBerry
X-OperaMini-Phone: RIM #
x-forwarded-for: 206.53.144.186, unknown

HTTP/1.1 200 OK
Date: Wed, 27 Aug 2008 17:16:08 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Language: en_us
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 2283
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=utf-8

GET /subsequent/request HTTP/1.1
User-Agent: Opera/9.50 (J2ME/MIDP; Opera Mini/4.1.11355/546; U; en)
Host: cookie.example.com
Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1
Accept-Language: en
Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0
Referer: http://cookie.example.com/
Cookie: cookie1=dpr07co4i5dchtgmk8g3gdt0b6; cookie3=1ce651a62bfe62a25faff44e5e5d500b; cookie2=en_us
Cookie2: $Version=1
Connection: Keep-Alive, TE
TE: deflate, gzip, chunked, identity, trailers
X-OperaMini-Features: advanced, file_system, folding
X-OperaMini-Phone-UA: BlackBerry
X-OperaMini-Phone: RIM #
x-forwarded-for: 206.53.144.186, unknown

HTTP/1.1 200 OK
Date: Wed, 27 Aug 2008 17:16:09 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Language: en_us
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 958
Keep-Alive: timeout=5, max=99
Connection: Keep-Alive
Content-Type: text/css

GET /image/example.gif HTTP/1.1
User-Agent: Opera/9.50 (J2ME/MIDP; Opera Mini/4.1.11355/546; U; en)
Host: cookie.example.com
Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1
Accept-Language: en
Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0
Referer: http://cookie.example.com/
If-Modified-Since: Wed, 20 Aug 2008 19:36:32 GMT
If-None-Match: "2c"
Cookie: cookie1=dpr07co4i5dchtgmk8g3gdt0b6; cookie3=1ce651a62bfe62a25faff44e5e5d500b; cookie2=en_us
Cookie2: $Version=1
Connection: Keep-Alive, TE
TE: deflate, gzip, chunked, identity, trailers
X-OperaMini-Features: advanced, file_system, folding
X-OperaMini-Phone-UA: BlackBerry
X-OperaMini-Phone: RIM #
x-forwarded-for: 206.53.144.186, unknown

HTTP/1.1 304 Not Modified
Date: Wed, 27 Aug 2008 17:16:10 GMT
Connection: Keep-Alive
Keep-Alive: timeout=5, max=98
ETag: "2c"

So what’s the solution in this situation? It looks like your best option is to set just one cookie—but shove all your values into it and delimit them in some way (and make sure it’s less than 1024 bytes). If you have multiple expire times, though, you will also have to handle expiration manually. Google does this:

Set-Cookie: PREF=ID=xxxxxxxxxxxxxxxx:TM=1219863220:LM=1219863220:S=yyyyyyyyyyyyyyyy; expires=Fri, 27-Aug-2010 18:53:40 GMT; path=/; domain=.google.com

Can everyone just buy an iPhone already?

Like this post? You might also like Coalmine, my centralized error tracking service for your apps. Coalmine captures errors and all kinds of helpful debugging information, notifies you, and makes it all searchable. Check it out!

Tags: ,

11 comments

  1. So I noticed this problem myself and it is due to the MDS or BIS service re-packaging the http headers as they arrive at the proxy. The HTTP specification requires on Set-Cookie Header element for each cookie set, but the BIS and MDS services for some god forsaken unknown reason re-processes the headers into a comma seperated list in a SINGLE Set-Cookie Header. The problem occurs if there is any comma in anywhere in your cookie it doesnt get parsed properly (in your case the ‘,’ in the expires field is probably screwing up the parser)

    Here is an example of what would be returned to the CLIENTdevice from BIS for you example above

    Set-Cookie: cookie1=32mrdio18lvpg6cduhnt1prmj5; path=/; domain=cookie.example.com, cookie2=en_us; expires=Thu, 27-Aug-2009 15:56:22 GMT; path=/; domain=cookie.example.com, cookie3=2d14a216211bc7f1a1b03fa747d1087e; expires=Mon, 01-Sep-2008 06:59:59 GMT; path=/; domain=cookie.example.com

    As you might imagine, comma seperating these values makes it fairly difficult to parse accuratly when commas appear in the content.

    Yes a huge bug, but not on the clients, on the BIS and MDS servers they use to route HTTP traffic to you, I wish they would just leave the content alone, causes me headaches.

  2. knight9: To be honest, I would welcome that being returned, but the truth is that it simply isn’t. In fact, sending multiple cookies in a single Set-Cookie header is perfectly fine according to RFC 2109 (http://www.ietf.org/rfc/rfc2109.txt):

    “[T]he Set-Cookie response header comprises the token Set-Cookie:, followed by a comma-separated list of one or more cookies. Each cookie begins with a NAME=VALUE pair, followed by zero or more semi-colon-separated attribute-value pairs.”

    Somewhere along the line (on the device or via a proxy) only one cookie is being returned, and sometimes even that is being garbled. This is indicative of a bigger problem:

    Cookie: cookie2=en_us
    Cookie: expires=Thu, 27-Aug-2009 15:57:03 GMT

    Serializing data into one cookie seems to be the only solution.

  3. I am hitting the same problem with the older BlackBerrys. I don’t know a single thing about the MDS and BIS stuff (and I am not impressed with RIM having to re-invent their own network stuff). But the Curve works fine though.

    Concatenating cookies isn’t an option for me as my ASP.Net server sends out other stuff in cookies too.

    I am surprised this problem is surfacing only now. Do BlackBerry users not browse the Internet?

    Is the Opera option viable? If not I will have to tell all my users to use the Nokia E71/E66.

    Thanks.

  4. Regarding comma separated cookies in the same line, I believe the cookie string has to be URL encoded. So real commas in the cookie should be %2c.

  5. HC: Well, thank goodness several manufacturers are starting to standardize on WebKit, at least. I can’t tell you the number of bizarre browser bugs I’ve ran across (on some Nokia phones, for example, you can’t use #FFFFFF—you can only get as close as #FFFFFE).

    Opera’s not really a viable option unless you have a captive or techie audience. It’s a fantastic browser for these phones, but not enough people know about it.

  6. I agree,cookies and rendering on the BlackBerry is absolutely horrible. Its like they didn’t even bother to think that the internet and web standards is important. They’re opening a store in the upcoming months but I’m sure it won’t touch Apple’s entire infrastructure from top to bottom

    - An extremely frustrated BlackBerry Developer

  7. It’s definitely a comma issue – if a blackberry makes a request having had multiple cookies successfully set, then it’ll send them comma concatenated. Causes no end of headaches.

  8. To be fair, the reinvented network is legacy from when RIM was sending email over the Mobitex network. The problem is not that they reinvented the network, it’s that it’s not necessary anymore and they still have it turned on all the time.

  9. I’m having the same problem, ie. second cookie not returned using the 8220 simulator.
    It’s an asp.net site, which has a sessionId cookie and a forms authentication cookie. Both under .net control, so contatenation isn’t reasonably an option.
    It isn’t a problem on the 8100 simulator or the 9500 simulator.

    Is this in fact a problem with the 4.6 OS?

  10. This seems to be an issue on the 9300 curve model. After logging in, the session does not hold when navigating to other pages.

  11. Amazing things here. I’m very satisfied to look your article.
    Thank you so much and I am having a look forward to touch you.
    Will you please drop me a mail?

Leave a comment