======= Cookies ======= Getting started =============== The cookies mapping has an extended mapping interface that allows getting, setting, and deleting the cookies that the browser is remembering for the current url, or for an explicitly provided URL. >>> from zope.testbrowser.testing import Browser >>> browser = Browser() Initially the browser does not point to a URL, and the cookies cannot be used. >>> len(browser.cookies) Traceback (most recent call last): ... RuntimeError: no request found >>> browser.cookies.keys() Traceback (most recent call last): ... RuntimeError: no request found Once you send the browser to a URL, the cookies attribute can be used. >>> browser.open('http://localhost/@@/testbrowser/simple.html') >>> len(browser.cookies) 0 >>> browser.cookies.keys() [] >>> browser.url 'http://localhost/@@/testbrowser/simple.html' >>> browser.cookies.url 'http://localhost/@@/testbrowser/simple.html' >>> import zope.testbrowser.interfaces >>> from zope.interface.verify import verifyObject >>> verifyObject(zope.testbrowser.interfaces.ICookies, browser.cookies) True Alternatively, you can use the ``forURL`` method to get another instance of the cookies mapping for the given URL. >>> len(browser.cookies.forURL('http://www.example.com')) 0 >>> browser.cookies.forURL('http://www.example.com').keys() [] >>> browser.cookies.forURL('http://www.example.com').url 'http://www.example.com' >>> browser.url 'http://localhost/@@/testbrowser/simple.html' >>> browser.cookies.url 'http://localhost/@@/testbrowser/simple.html' Here, we use a view that will make the server set cookies with the values we provide. >>> browser.open('http://localhost/set_cookie.html?name=foo&value=bar') >>> browser.headers['set-cookie'].replace(';', '') 'foo=bar' Basic Mapping Interface ======================= Now the cookies for localhost have a value. These are examples of just the basic accessor operators and methods. >>> browser.cookies['foo'] 'bar' >>> browser.cookies.keys() ['foo'] >>> browser.cookies.values() ['bar'] >>> browser.cookies.items() [('foo', 'bar')] >>> 'foo' in browser.cookies True >>> 'bar' in browser.cookies False >>> len(browser.cookies) 1 >>> print(dict(browser.cookies)) {'foo': 'bar'} As you would expect, the cookies attribute can also be used to examine cookies that have already been set in a previous request. To demonstrate this, we use another view that does not set cookies but reports on the cookies it receives from the browser. >>> browser.open('http://localhost/get_cookie.html') >>> print browser.headers.get('set-cookie') None >>> browser.contents 'foo: bar' >>> browser.cookies['foo'] 'bar' The standard mapping mutation methods and operators are also available, as seen here. >>> browser.cookies['sha'] = 'zam' >>> len(browser.cookies) 2 >>> import pprint >>> pprint.pprint(sorted(browser.cookies.items())) [('foo', 'bar'), ('sha', 'zam')] >>> browser.open('http://localhost/get_cookie.html') >>> print browser.headers.get('set-cookie') None >>> print browser.contents # server got the cookie change foo: bar sha: zam >>> browser.cookies.update({'va': 'voom', 'tweedle': 'dee'}) >>> pprint.pprint(sorted(browser.cookies.items())) [('foo', 'bar'), ('sha', 'zam'), ('tweedle', 'dee'), ('va', 'voom')] >>> browser.open('http://localhost/get_cookie.html') >>> print browser.headers.get('set-cookie') None >>> print browser.contents foo: bar sha: zam tweedle: dee va: voom >>> del browser.cookies['foo'] >>> del browser.cookies['tweedle'] >>> browser.open('http://localhost/get_cookie.html') >>> print browser.contents sha: zam va: voom Headers ======= You can see the Cookies header that will be sent to the browser in the ``header`` attribute and the repr and str. >>> browser.cookies.header 'sha=zam; va=voom' >>> browser.cookies # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE >>> str(browser.cookies) 'sha=zam; va=voom' Extended Mapping Interface ========================== ------------------------------------------ Read Methods: ``getinfo`` and ``iterinfo`` ------------------------------------------ ``getinfo`` ----------- The ``cookies`` mapping also has an extended interface to get and set extra information about each cookie. ``getinfo`` returns a dictionary. Here is the interface description. :: def getinfo(name): """returns dict of settings for the given cookie name. This includes only the following cookie values: - name (str) - value (str), - port (int or None), - domain (str), - path (str or None), - secure (bool), and - expires (datetime.datetime with pytz.UTC timezone or None), - comment (str or None), - commenturl (str or None). """ Here are some examples. >>> browser.open('http://localhost/set_cookie.html?name=foo&value=bar') >>> pprint.pprint(browser.cookies.getinfo('foo')) {'comment': None, 'commenturl': None, 'domain': 'localhost.local', 'expires': None, 'name': 'foo', 'path': '/', 'port': None, 'secure': False, 'value': 'bar'} >>> pprint.pprint(browser.cookies.getinfo('sha')) {'comment': None, 'commenturl': None, 'domain': 'localhost.local', 'expires': None, 'name': 'sha', 'path': '/', 'port': None, 'secure': False, 'value': 'zam'} >>> import datetime >>> expires = datetime.datetime(2030, 1, 1).strftime( ... '%a, %d %b %Y %H:%M:%S GMT') >>> browser.open( ... 'http://localhost/set_cookie.html?name=wow&value=wee&' ... 'expires=%s' % ... (expires,)) >>> pprint.pprint(browser.cookies.getinfo('wow')) {'comment': None, 'commenturl': None, 'domain': 'localhost.local', 'expires': datetime.datetime(2030, 1, 1, 0, 0, tzinfo=), 'name': 'wow', 'path': '/', 'port': None, 'secure': False, 'value': 'wee'} Max-age is converted to an "expires" value. >>> browser.open( ... 'http://localhost/set_cookie.html?name=max&value=min&' ... 'max-age=3000&&comment=silly+billy') >>> pprint.pprint(browser.cookies.getinfo('max')) # doctest: +ELLIPSIS {'comment': 'silly%20billy', 'commenturl': None, 'domain': 'localhost.local', 'expires': datetime.datetime(..., tzinfo=), 'name': 'max', 'path': '/', 'port': None, 'secure': False, 'value': 'min'} ``iterinfo`` ------------ You can iterate over all of the information about the cookies for the current page using the ``iterinfo`` method. >>> pprint.pprint(sorted(browser.cookies.iterinfo(), ... key=lambda info: info['name'])) ... # doctest: +ELLIPSIS [{'comment': None, 'commenturl': None, 'domain': 'localhost.local', 'expires': None, 'name': 'foo', 'path': '/', 'port': None, 'secure': False, 'value': 'bar'}, {'comment': 'silly%20billy', 'commenturl': None, 'domain': 'localhost.local', 'expires': datetime.datetime(..., tzinfo=), 'name': 'max', 'path': '/', 'port': None, 'secure': False, 'value': 'min'}, {'comment': None, 'commenturl': None, 'domain': 'localhost.local', 'expires': None, 'name': 'sha', 'path': '/', 'port': None, 'secure': False, 'value': 'zam'}, {'comment': None, 'commenturl': None, 'domain': 'localhost.local', 'expires': None, 'name': 'va', 'path': '/', 'port': None, 'secure': False, 'value': 'voom'}, {'comment': None, 'commenturl': None, 'domain': 'localhost.local', 'expires': datetime.datetime(2030, 1, 1, 0, 0, tzinfo=), 'name': 'wow', 'path': '/', 'port': None, 'secure': False, 'value': 'wee'}] Extended Examples ----------------- If you want to look at the cookies for another page, you can either navigate to the other page in the browser, or, as already mentioned, you can use the ``forURL`` method, which returns an ICookies instance for the new URL. >>> sorted(browser.cookies.forURL( ... 'http://localhost/inner/set_cookie.html').keys()) ['foo', 'max', 'sha', 'va', 'wow'] >>> extra_cookie = browser.cookies.forURL( ... 'http://localhost/inner/set_cookie.html') >>> extra_cookie['gew'] = 'gaw' >>> extra_cookie.getinfo('gew')['path'] '/inner' >>> sorted(extra_cookie.keys()) ['foo', 'gew', 'max', 'sha', 'va', 'wow'] >>> sorted(browser.cookies.keys()) ['foo', 'max', 'sha', 'va', 'wow'] >>> import zope.site.folder >>> getRootFolder()['inner'] = zope.site.folder.Folder() >>> getRootFolder()['inner']['path'] = zope.site.folder.Folder() >>> import transaction >>> transaction.commit() >>> browser.open('http://localhost/inner/get_cookie.html') >>> print browser.contents # has gewgaw foo: bar gew: gaw max: min sha: zam va: voom wow: wee >>> browser.open('http://localhost/inner/path/get_cookie.html') >>> print browser.contents # has gewgaw foo: bar gew: gaw max: min sha: zam va: voom wow: wee >>> browser.open('http://localhost/get_cookie.html') >>> print browser.contents # NO gewgaw foo: bar max: min sha: zam va: voom wow: wee Here's an example of the server setting a cookie that is only available on an inner page. >>> browser.open( ... 'http://localhost/inner/path/set_cookie.html?name=big&value=kahuna' ... ) >>> browser.cookies['big'] 'kahuna' >>> browser.cookies.getinfo('big')['path'] '/inner/path' >>> browser.cookies.getinfo('gew')['path'] '/inner' >>> browser.cookies.getinfo('foo')['path'] '/' >>> print browser.cookies.forURL('http://localhost/').get('big') None ---------------------------------------- Write Methods: ``create`` and ``change`` ---------------------------------------- The basic mapping API only allows setting values. If a cookie already exists for the given name, it's value will be changed; or else a new cookie will be created for the current request's domain and a path of '/', set to last for only this browser session (a "session" cookie). To create or change cookies with different additional information, use the ``create`` and ``change`` methods, respectively. Here is an example of ``create``. >>> from pytz import UTC >>> browser.cookies.create( ... 'bling', value='blang', path='/inner', ... expires=datetime.datetime(2020, 1, 1, tzinfo=UTC), ... comment='follow swallow') >>> pprint.pprint(browser.cookies.getinfo('bling')) {'comment': 'follow%20swallow', 'commenturl': None, 'domain': 'localhost.local', 'expires': datetime.datetime(2020, 1, 1, 0, 0, tzinfo=), 'name': 'bling', 'path': '/inner', 'port': None, 'secure': False, 'value': 'blang'} In these further examples of ``create``, note that the testbrowser sends all domains to Zope, and both http and https. >>> browser.open('https://dev.example.com/inner/path/get_cookie.html') >>> browser.cookies.keys() # a different domain [] >>> browser.cookies.create('tweedle', 'dee') >>> pprint.pprint(browser.cookies.getinfo('tweedle')) {'comment': None, 'commenturl': None, 'domain': 'dev.example.com', 'expires': None, 'name': 'tweedle', 'path': '/inner/path', 'port': None, 'secure': False, 'value': 'dee'} >>> browser.cookies.create( ... 'boo', 'yah', domain='.example.com', path='/inner', secure=True) >>> pprint.pprint(browser.cookies.getinfo('boo')) {'comment': None, 'commenturl': None, 'domain': '.example.com', 'expires': None, 'name': 'boo', 'path': '/inner', 'port': None, 'secure': True, 'value': 'yah'} >>> sorted(browser.cookies.keys()) ['boo', 'tweedle'] >>> browser.open('https://dev.example.com/inner/path/get_cookie.html') >>> print browser.contents boo: yah tweedle: dee >>> browser.open( # not https, so not secure, so not 'boo' ... 'http://dev.example.com/inner/path/get_cookie.html') >>> sorted(browser.cookies.keys()) ['tweedle'] >>> print browser.contents tweedle: dee >>> browser.open( # not tweedle's domain ... 'https://prod.example.com/inner/path/get_cookie.html') >>> sorted(browser.cookies.keys()) ['boo'] >>> print browser.contents boo: yah >>> browser.open( # not tweedle's domain ... 'https://example.com/inner/path/get_cookie.html') >>> sorted(browser.cookies.keys()) ['boo'] >>> print browser.contents boo: yah >>> browser.open( # not tweedle's path ... 'https://dev.example.com/inner/get_cookie.html') >>> sorted(browser.cookies.keys()) ['boo'] >>> print browser.contents boo: yah Masking by Path --------------- The API allows creation of cookies that mask existing cookies, but it does not allow creating a cookie that will be immediately masked upon creation. Having multiple cookies with the same name for a given URL is rare, and is a pathological case for using a mapping API to work with cookies, but it is supported to some degree, as demonstrated below. Note that the Cookie RFCs (2109, 2965) specify that all matching cookies be sent to the server, but with an ordering so that more specific paths come first. We also prefer more specific domains, though the RFCs state that the ordering of cookies with the same path is indeterminate. The best-matching cookie is the one that the mapping API uses. Also note that ports, as sent by RFC 2965's Cookie2 and Set-Cookie2 headers, are parsed and stored by this API but are not used for filtering as of this writing. This is an example of making one cookie that masks another because of path. First, unless you pass an explicit path, you will be modifying the existing cookie. >>> browser.open('https://dev.example.com/inner/path/get_cookie.html') >>> print browser.contents boo: yah tweedle: dee >>> browser.cookies.getinfo('boo')['path'] '/inner' >>> browser.cookies['boo'] = 'hoo' >>> browser.cookies.getinfo('boo')['path'] '/inner' >>> browser.cookies.getinfo('boo')['secure'] True Now we mask the cookie, using the path. >>> browser.cookies.create('boo', 'boo', path='/inner/path') >>> browser.cookies['boo'] 'boo' >>> browser.cookies.getinfo('boo')['path'] '/inner/path' >>> browser.cookies.getinfo('boo')['secure'] False >>> browser.cookies['boo'] 'boo' >>> sorted(browser.cookies.keys()) ['boo', 'tweedle'] To identify the additional cookies, you can change the URL... >>> extra_cookies = browser.cookies.forURL( ... 'https://dev.example.com/inner/get_cookie.html') >>> extra_cookies['boo'] 'hoo' >>> extra_cookies.getinfo('boo')['path'] '/inner' >>> extra_cookies.getinfo('boo')['secure'] True ...or use ``iterinfo`` and pass in a name. >>> pprint.pprint(list(browser.cookies.iterinfo('boo'))) [{'comment': None, 'commenturl': None, 'domain': 'dev.example.com', 'expires': None, 'name': 'boo', 'path': '/inner/path', 'port': None, 'secure': False, 'value': 'boo'}, {'comment': None, 'commenturl': None, 'domain': '.example.com', 'expires': None, 'name': 'boo', 'path': '/inner', 'port': None, 'secure': True, 'value': 'hoo'}] An odd situation in this case is that deleting a cookie can sometimes reveal another one. >>> browser.open('https://dev.example.com/inner/path/get_cookie.html') >>> browser.cookies['boo'] 'boo' >>> del browser.cookies['boo'] >>> browser.cookies['boo'] 'hoo' Creating a cookie that will be immediately masked within the current url is not allowed. >>> browser.cookies.getinfo('tweedle')['path'] '/inner/path' >>> browser.cookies.create('tweedle', 'dum', path='/inner') ... # doctest: +NORMALIZE_WHITESPACE Traceback (most recent call last): ... ValueError: cannot set a cookie that will be hidden by another cookie for this url (https://dev.example.com/inner/path/get_cookie.html) >>> browser.cookies['tweedle'] 'dee' Masking by Domain ----------------- All of the same behavior is also true for domains. The only difference is a theoretical one: while the behavior of masking cookies via paths is defined by the relevant IRCs, it is not defined for domains. Here, we simply follow a "best match" policy. We initialize by setting some cookies for example.org. >>> browser.open('https://dev.example.org/get_cookie.html') >>> browser.cookies.keys() # a different domain [] >>> browser.cookies.create('tweedle', 'dee') >>> browser.cookies.create('boo', 'yah', domain='example.org', ... secure=True) Before we look at the examples, note that the default behavior of the cookies is to be liberal in the matching of domains. >>> browser.cookies.strict_domain_policy False According to the RFCs, a domain of 'example.com' can only be set implicitly from the server, and implies an exact match, so example.com URLs will get the cookie, but not *.example.com (i.e., dev.example.com). Real browsers vary in their behavior in this regard. The cookies collection, by default, has a looser interpretation of this, such that domains are always interpreted as effectively beginning with a ".", so dev.example.com will include a cookie from the "example.com" domain filter as if it were a ".example.com" filter. Here's an example. If we go to dev.example.org, we should only see the "tweedle" cookie if we are using strict rules. But right now we are using loose rules, so 'boo' is around too. >>> browser.open('https://dev.example.org/get_cookie.html') >>> sorted(browser.cookies) ['boo', 'tweedle'] >>> print browser.contents boo: yah tweedle: dee If we set strict_domain_policy to True, then only tweedle is included. >>> browser.cookies.strict_domain_policy = True >>> sorted(browser.cookies) ['tweedle'] >>> browser.open('https://dev.example.org/get_cookie.html') >>> print browser.contents tweedle: dee If we set the "boo" domain to ".example.org" (as it would be set under the more recent Cookie RFC if a server sent the value) then maybe we get the "boo" value again. >>> browser.cookies.forURL('https://example.org').change( ... 'boo', domain=".example.org") Traceback (most recent call last): ... ValueError: policy does not allow this cookie Whoa! Why couldn't we do that? Well, the strict_domain_policy affects what cookies we can set also. With strict rules, ".example.org" can only be set by "*.example.org" domains, *not* example.org itself. OK, we'll create a new cookie then. >>> browser.cookies.forURL('https://snoo.example.org').create( ... 'snoo', 'kums', domain=".example.org") >>> sorted(browser.cookies) ['snoo', 'tweedle'] >>> browser.open('https://dev.example.org/get_cookie.html') >>> print browser.contents snoo: kums tweedle: dee Let's set things back to the way they were. >>> del browser.cookies['snoo'] >>> browser.cookies.strict_domain_policy = False >>> browser.open('https://dev.example.org/get_cookie.html') >>> sorted(browser.cookies) ['boo', 'tweedle'] >>> print browser.contents boo: yah tweedle: dee Now back to the the examples of masking by domain. First, unless you pass an explicit domain, you will be modifying the existing cookie. >>> browser.cookies.getinfo('boo')['domain'] 'example.org' >>> browser.cookies['boo'] = 'hoo' >>> browser.cookies.getinfo('boo')['domain'] 'example.org' >>> browser.cookies.getinfo('boo')['secure'] True Now we mask the cookie, using the domain. >>> browser.cookies.create('boo', 'boo', domain='dev.example.org') >>> browser.cookies['boo'] 'boo' >>> browser.cookies.getinfo('boo')['domain'] 'dev.example.org' >>> browser.cookies.getinfo('boo')['secure'] False >>> browser.cookies['boo'] 'boo' >>> sorted(browser.cookies.keys()) ['boo', 'tweedle'] To identify the additional cookies, you can change the URL... >>> extra_cookies = browser.cookies.forURL( ... 'https://example.org/get_cookie.html') >>> extra_cookies['boo'] 'hoo' >>> extra_cookies.getinfo('boo')['domain'] 'example.org' >>> extra_cookies.getinfo('boo')['secure'] True ...or use ``iterinfo`` and pass in a name. >>> pprint.pprint(list(browser.cookies.iterinfo('boo'))) [{'comment': None, 'commenturl': None, 'domain': 'dev.example.org', 'expires': None, 'name': 'boo', 'path': '/', 'port': None, 'secure': False, 'value': 'boo'}, {'comment': None, 'commenturl': None, 'domain': 'example.org', 'expires': None, 'name': 'boo', 'path': '/', 'port': None, 'secure': True, 'value': 'hoo'}] An odd situation in this case is that deleting a cookie can sometimes reveal another one. >>> browser.open('https://dev.example.org/get_cookie.html') >>> browser.cookies['boo'] 'boo' >>> del browser.cookies['boo'] >>> browser.cookies['boo'] 'hoo' Setting a cookie with a foreign domain from the current URL is not allowed (use forURL to get around this). >>> browser.cookies.create('tweedle', 'dum', domain='locahost.local') Traceback (most recent call last): ... ValueError: current url must match given domain >>> browser.cookies['tweedle'] 'dee' Setting a cookie that will be immediately masked within the current url is also not allowed. >>> browser.cookies.getinfo('tweedle')['domain'] 'dev.example.org' >>> browser.cookies.create('tweedle', 'dum', domain='.example.org') ... # doctest: +NORMALIZE_WHITESPACE Traceback (most recent call last): ... ValueError: cannot set a cookie that will be hidden by another cookie for this url (https://dev.example.org/get_cookie.html) >>> browser.cookies['tweedle'] 'dee' ``change`` ---------- So far all of our examples in this section have centered on ``create``. ``change`` allows making changes to existing cookies. Changing expiration is a good example. >>> browser.open("http://localhost/@@/testbrowser/cookies.html") >>> browser.cookies['foo'] = 'bar' >>> browser.cookies.change('foo', expires=datetime.datetime(2021, 1, 1)) >>> browser.cookies.getinfo('foo')['expires'] datetime.datetime(2021, 1, 1, 0, 0, tzinfo=) That's the main story. Now here are some edge cases. >>> browser.cookies.change( ... 'foo', ... expires=zope.testbrowser.cookies.expiration_string( ... datetime.datetime(2020, 1, 1))) >>> browser.cookies.getinfo('foo')['expires'] datetime.datetime(2020, 1, 1, 0, 0, tzinfo=) >>> browser.cookies.forURL( ... 'http://localhost/@@/testbrowser/cookies.html').change( ... 'foo', ... expires=zope.testbrowser.cookies.expiration_string( ... datetime.datetime(2019, 1, 1))) >>> browser.cookies.getinfo('foo')['expires'] datetime.datetime(2019, 1, 1, 0, 0, tzinfo=) >>> browser.cookies['foo'] 'bar' >>> browser.cookies.change('foo', expires=datetime.datetime(1999, 1, 1)) >>> len(browser.cookies) 4 While we are at it, it is worth noting that trying to create a cookie that has already expired raises an error. >>> browser.cookies.create('foo', 'bar', ... expires=datetime.datetime(1999, 1, 1)) Traceback (most recent call last): ... AlreadyExpiredError: May not create a cookie that is immediately expired Clearing cookies ---------------- clear, clearAll, clearAllSession allow various clears of the cookies. The ``clear`` method clears all of the cookies for the current page. >>> browser.open('http://localhost/@@/testbrowser/cookies.html') >>> len(browser.cookies) 4 >>> browser.cookies.clear() >>> len(browser.cookies) 0 The ``clearAllSession`` method clears *all* session cookies (for all domains and paths, not just the current URL), as if the browser had been restarted. >>> browser.cookies.clearAllSession() >>> len(browser.cookies) 0 The ``clearAll`` removes all cookies for the browser. >>> browser.cookies.clearAll() >>> len(browser.cookies) 0 Note that explicitly setting a Cookie header is an error if the ``cookies`` mapping has any values; and adding a new cookie to the ``cookies`` mapping is an error if the Cookie header is already set. This is to prevent hard-to- diagnose intermittent errors when one header or the other wins. >>> browser.cookies['boo'] = 'yah' >>> browser.addHeader('Cookie', 'gee=gaw') Traceback (most recent call last): ... ValueError: cookies are already set in `cookies` attribute >>> browser.cookies.clearAll() >>> browser.addHeader('Cookie', 'gee=gaw') >>> browser.cookies['fee'] = 'fi' Traceback (most recent call last): ... ValueError: cookies are already set in `Cookie` header