Merge pull request #48 from resync/develop

Release 2.0.1
Change master->main
2021-03-23 08:24:21 -04:00 · 2021-03-23 08:21:41 -04:00 · 2021-03-23 08:15:42 -04:00 · 2021-03-23 07:59:32 -04:00 · 2021-03-22 09:33:33 -04:00 · 2021-03-18 22:00:01 -04:00
154 changed files with 4015 additions and 2111 deletions
--- a/.gitignore
+++ b/.gitignore
@ -2,6 +2,7 @@
 build
 dist
 MANIFEST
+.tox
 .eggs
 *.egg
 # Testing
@ -10,10 +11,8 @@ htmlcov
 # Files generated by client / local config
 .resync-client-status.cfg
 resync-client.log
-resync.library.cornell.edu_sim100
-resync.library.cornell.edu_sim1000
-resync.library.cornell.edu_sim10000
 localhost_8888
+tmp
 # Related to packaging
 resync.egg-info
 # Other...
@ -21,5 +20,4 @@ resync.egg-info
 *.pyc
 *~
 .cache
-rc
-re
+
--- a/.travis.yml
+++ b/.travis.yml
@ -1,21 +1,16 @@
 language: python
 python:
-#  - "2.6" -- commenting until https://github.com/resync/resync/issues/23 fixed
-  - "2.7"
-  - "3.3"
-  - "3.4"
  - "3.5"
  - "3.6"
+  - "3.7"
+  - "3.8"
 install:
-  - pip install requests
-  - pip install python-dateutil
-  - pip install coveralls pep8 pep257 restructuredtext_lint testfixtures
+  - pip install coveralls pycodestyle pep257 testfixtures
  - python setup.py install
 script:
  - python setup.py test
-  - pep8 --ignore=E501 resync tests bin/resync bin/resync-explorer
-  - pep257 resync
-  - rst-lint README
+  - pycodestyle --ignore=E501,W503 resync tests resync-sync resync-build resync-explorer
+  - pep257 resync bin tests resync-sync resync-build resync-explorer
  - coverage run --source=resync setup.py test
 after_success:
  - coveralls
--- a/CHANGES.md
+++ b/CHANGES.md
@ -1,14 +1,35 @@
-resync change log
-=================
+# resync change log

-The first two components of the version tags are tied to the ResourceSync 
-core specification version. Versions 1.0.x implement the v1.0 
-ResourceSync specification which was standardized as ANSI/NISO Z39.99-2014
-<http://www.openarchives.org/rs/1.0/toc>.
+v2.0.1 2021-03-23
+  * Route all URI and file requests through `resync/url_or_file_open.py` so that settings such as authentication headers can be consistently applied
+  * Do not exclude any directories from sync by default, specify with --exclude
+  * Fix comparison of using possibly multiple checksums
+  * Improve test coverage
+  * Remove Python 2 cruft

-v1.0.8 2017-??-??
+v2.0.0 2020-12-16
+  * Supports ResourceSync v1.1 ANSI/NISO Z39.99-2017 <http://www.openarchives.org/rs/1.1/toc> as default
+  * Support for the prior v1.0 ANSI/NISO Z39.99-2014 <http://www.openarchives.org/rs/1.0/toc> is retained with `spec-version='1.0'` option in scripts and `spec_version='1.0'` in various classes
+  * Split old `resync` script into `resync-sync` and `resync-build`
+  * Move scripts from `bin` dir to base dir for easier testing/development (now that there is no conflict with the module name)
+  * Switch from optparse to argparse, use exclusive argument group for commands
+  * Add --access_token option to pass bearer token with web requests
+  * Add --delay option to pause between successive web requests
+  * Drop Python 2.7, 3.3 & 3.4 from tests, add 3.7 & 3.8
+  * Fix various depracation warnings and errors for 3.7 & 3.8
+  * Switch from pep8 to pycodestyle in tests
+  * Change README from RDS to github flavored Markdown
+  * Move libraries to support tests into test/testlib
+
+v1.0.9 2018-10-23
+  * Add -t/--tries and -T/--timeout options (https://github.com/resync/resync/issues/34)
+  * Includes partial replacement of urllib.request with requests
+
+v1.0.8 2017-08-30
+  * Fix support for explicitly writing ResourceList objects as indexes (https://github.com/resync/resync/issues/31)
  * Change argument names for local modes to have `--write-` prefix, e.g. --write-resourcelist
  * Improve modes to detect capabilities
+  * Abandon support for Python 2.6 (https://github.com/resync/resync/issues/23)

 v1.0.7 2017-03-27
  * Fixed silly error in `resync-explorer` introduced in 1.0.6
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@ -1,6 +1,6 @@
 # Contributing to resync

-Issues and pull requests are appreciated.
+Issues and pull requests are appreciated. I apologize if it takes me some time to respond to issues.

 ## Submitting issues

@ -12,8 +12,13 @@ If you suggest a new feature, please give an example of expected behavior and us

 If submitting a pull request:

+   * Understand that this code and any merged contributions are covered by an Apache 2.0 license
   * Please discuss in a issue before submitting a pull request for significant changes.
+   * Please submit pull requests against the `develop` branch (changes are collected there before making a release).
   * Please follow [PEP8](https://www.python.org/dev/peps/pep-0008/) and [PEP257](https://www.python.org/dev/peps/pep-0257/) style rules.
   * Please don't repeat code.
   * Please cover the code with tests.

+## Local conventions
+
+   * Tests: Use `from tests.testcase_with_tmpdir import TestCase` in place of `unittest.TestCase` to get a test object with `self.tmpdir` as temporary location to write to.
--- a/MANIFEST.in
+++ b/MANIFEST.in
@ -1,2 +0,0 @@
-include *.txt
-recursive-include docs *.txt
--- a/123
+++ b/123
@ -1,123 +0,0 @@
-======
-resync
-======
-
-.. image:: https://travis-ci.org/resync/resync.png?branch=master
-  :target: https://travis-ci.org/resync/resync
-  :alt: Build status
-
-.. image:: https://coveralls.io/repos/github/resync/resync/badge.svg?branch=master
-  :target: https://coveralls.io/github/resync/resync
-  :alt: Test coverage
-
-**resync** is a ResourceSync client and library in python. `ResourceSync 
-<http://www.openarchives.org/rs/>`_ is a synchronization framework for 
-the web consisting of various capabilities that allow third party 
-systems to remain synchronized with a server's evolving resources. 
-The capabilities can be combined in a modular manner to meet local 
-or community requirements.
-
-Client usage
------------
-
-Typical client usage to synchronize from a source at 
-``http://source.example.com/`` to a set of local files would be::
-
-    resync http://source.example.com/
-
-which will create or update a local directory ``./source.example.com``. 
-Alternatively, the destination directory may be specified explicitly::
-
-    resync http://source.example.com/ /tmp/my_copy
-
-Option details and a number of different modes are described with::
-
-    resync -h
-
-Library usage
-------------
-
-Typical library use in a source (create and output a Resource List)::
-
-    from resync import Resource,ResourceList
-
-    rl = ResourceList()
-    rl.add( Resource('http://example.com/res1', lastmod='2013-01-01') )
-    rl.add( Resource('http://example.com/res2', lastmod='2013-01-02') )
-    print rl.as_xml()
-
-Typical library use in a destination (get and examine a Capability List):: 
-
-    from resync import CapabilityList
-
-    # Read Capability List and show supported capabilities
-    cl = CapabilityList()
-    cl.read("https://raw.github.com/resync/resync/1.0/resync/test/testdata/examples_from_spec/resourcesync_ex_2_6.xml")
-    for resource in cl:
-        print "supports %s (at %s)" % (resource.capability,resource.uri)
-
-Installation
------------
-
-The client and library are designed to work with Python 2.6, 2.7, 3.4, 3.5 and 3.6.
-
-**Automatic installation**::
-
-    easy_install resync
-
-rsync is listed in `PyPI
-<http://pypi.python.org/pypi/resync>`_ and can be installed with 
-``pip`` or ``easy_install``. 
-
-**Manual installation from github**::
-
-    cd /tmp
-    git clone git://github.com/resync/resync.git
-    cd resync/
-    python setup.py build
-    sudo python setup.py install
-
-This will install the library code in the appropriate place within 
-your python setup, and the client ``resync`` in an appropriate system 
-path (perhaps ``/usr/local/bin`` or ``/usr/bin`` depending on your system).
-
-The source code is maintained on `Github
-<https://github.com/resync/resync>`_
-and there may be branches/versions available there that are not 
-yet packaged for PyPI.
-
-See also
--------
-
-`ResourceSync simulator
-<http://github.com/resync/simulator>`_
-
-Contributors
------------
-
-Thanks to:
-`Bernhard Haslhofer
-<https://github.com/behas>`_,
-`Robert Sanderson
-<https://github.com/azaroth42>`_
-
-See also CONTRIBUTING.md
-
-Copyright and License
---------------------
-
-Copyright 2012--2017 Simeon Warner
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-   http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-   
-See LICENSE.txt
--- a/README.md
+++ b/README.md
@ -0,0 +1,124 @@
+# resync
+
+[![Build Status](https://travis-ci.org/resync/resync.svg?branch=main)](https://travis-ci.org/resync/resync)
+[![Test Coverage](https://coveralls.io/repos/github/resync/resync/badge.svg?branch=main)](https://coveralls.io/github/resync/resync)
+
+**resync** is a ResourceSync library with supporting client scipts,
+written in python.
+[ResourceSync](http://www.openarchives.org/rs/) is a synchronization
+framework for the web consisting of various capabilities that allow
+third party systems to remain synchronized with a server's evolving
+resources. The capabilities can be combined in a modular manner to
+meet local or community requirements.
+
+## Client usage
+
+Typical client usage to synchronize from a source at
+`http://source.example.com/` to a set of local files would be
+
+```
+resync-sync http://source.example.com/
+```
+
+which will create or update a local directory `./source.example.com`.
+Alternatively, the destination directory may be specified explicitly::
+
+```
+resync-sync http://source.example.com/ /tmp/my_copy
+```
+
+Option details and a number of different modes are described with::
+
+```
+resync-sync -h
+```
+
+## Python library usage
+
+Typical library use in a source (create and output a Resource List)::
+
+```
+from resync import Resource, ResourceList
+
+rl = ResourceList()
+rl.add( Resource('http://example.com/res1', lastmod='2013-01-01') )
+rl.add( Resource('http://example.com/res2', lastmod='2013-01-02') )
+print rl.as_xml()
+```
+
+Typical library use in a destination (get and examine a Capability List)::
+
+```
+from resync import CapabilityList
+
+# Read Capability List and show supported capabilities
+cl = CapabilityList()
+cl.read("https://raw.github.com/resync/resync/1.0/resync/test/testdata/examples_from_spec/resourcesync_ex_2_6.xml")
+for resource in cl:
+    print "supports %s (at %s)" % (resource.capability,resource.uri)
+```
+
+## Installation
+
+The client and library are designed to work with Python 3.5 and up. (The
+last version supporting Python 2.7 was
+[1.0.9, also on PyPI](https://pypi.org/project/resync/1.0.9/))
+
+### Automatic installation
+
+```
+pip install resync
+```
+
+rsync is listed in [PyPI](http://pypi.python.org/pypi/resync) and can be
+installed with `pip` or `easy_install`.
+
+### Manual installation from github
+
+```
+cd /tmp
+git clone git://github.com/resync/resync.git
+cd resync/
+python setup.py build
+python setup.py install
+```
+
+This will install the library code in the appropriate place within
+a user-space python setup, including the clients `resync-sync`,
+`resync-build` and `resync-explorer`. Use of `sudo` to install in
+system spaces is generally discouraged.
+
+The source code is maintained on [Github](https://github.com/resync/resync)
+and there may be branches/versions available there that are not
+yet packaged for PyPI.
+
+## See also
+
+  * [ResourceSync simulator](http://github.com/resync/simulator)
+
+## Contributors
+
+Thanks to: [Bernhard Haslhofer](https://github.com/behas),
+[Robert Sanderson](<https://github.com/azaroth42)
+and [other contributors](https://github.com/resync/resync/graphs/contributors).
+
+See [CONTRIBUTING.md](https://github.com/resync/resync/blob/main/CONTRIBUTING.md)
+for guidelines for contributing.
+
+## Copyright and License
+
+Copyright 2012--2020 Simeon Warner and contributors
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+
+See `LICENSE.txt`
--- a/README.rst
+++ b/README.rst
@ -1 +0,0 @@
-README
--- a/bin/resync
+++ b/bin/resync
@ -1,293 +0,0 @@
-#!/usr/bin/env python
-"""
-resync: The ResourceSync command line client
-
-Copyright 2012,2013 Simeon Warner
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License
-"""
-
-import optparse
-import sys
-
-from resync import __version__
-from resync.client import Client, ClientFatalError
-from resync.client_utils import init_logging, count_true_args, parse_links, parse_capabilities, parse_capability_lists
-
-DEFAULT_LOGFILE = 'resync-client.log'
-
-
-def main():
-
-    if (sys.version_info < (2, 6)):
-        sys.exit("This program requires python version 2.6 or later")
-
-    # Options and arguments
-    p = optparse.OptionParser(description='ResourceSync command line client',
-                              usage='usage: %prog [options] uri_path local_path  (-h for help)',
-                              version='%prog ' + __version__)
-
-    # Modes
-    # a. modes using remote sitemap/resources
-    rem = p.add_option_group('REMOTE MODES',
-                             'These modes use a remote source that is specified in a set of uri=path mappings '
-                             'and potentially also using an explicit --sitemap location. The default mode is '
-                             '--baseline. See also: resync-explorer for an interactive client.')
-    rem.add_option('--baseline', '-b', action='store_true',
-                   help='baseline sync of resources from remote source (src) to local filesystem (dst)')
-    rem.add_option('--incremental', '--inc', '-i', action='store_true',
-                   help='incremental sync of resources from remote source (src) to local filesystem (dst). Uses either timestamp recorded from last baseline or incremental sync for this source, or explicit --from parameter, to determine the earlier update timestamp to act on.')
-    rem.add_option('--audit', '-a', action='store_true',
-                   help="audit sync state of destination wrt source")
-    rem.add_option('--parse', '-p', action='store_true',
-                   help="parse a remote sitemap/sitemapindex (from mapping or explicit --sitemap) and show summary information including document type and number of entries")
-    # b. modes based solely on files on local disk
-    loc = p.add_option_group('LOCAL MODES',
-                             'These modes act on files on the local disk')
-    loc.add_option('--write-resourcelist', '--write-resource-list', action='store_true',
-                   help="write a resource list based on files on disk using uri=path mappings "
-                        "in reverse to calculate URIs from the local paths. Scans local disk "
-                        "based either on explicit --paths setting, else starting from all local "
-                        "paths specified in the mappings. Writes to STDOUT by default, override "
-                        "with --outfile")
-    loc.add_option('--write-changelist', '--write-change-list', action='store_true',
-                   help="write a change list based on comparison of a reference sitemap "
-                        "(specify file with --reference) and either files on disk (using "
-                        "the mapping provided) or a second sitemap (specify file with "
-                        "--newreference). Otherwise follows --write-resourcelist options. Also accepts "
-                        "the --empty option (with no mapping) to write and empty changelist.")
-    loc.add_option('--write-capabilitylist', '--write-capability-list', type=str, action='store',
-                   help="write a capability list based on the set of capabilities and "
-                        "URIs supplied in cap_name=URI,cap_name=URI format. Otherwise "
-                        "follows --write-resourcelist options.")
-    loc.add_option('--write-sourcedescription', '--write-source-description', type=str, action='store',
-                   help="write a Source Description document based on the set of capability "
-                        "list URIs supplied as a comma separated list. Otherwise "
-                        "follows --write-resourcelist options.")
-    loc.add_option('--write-resourcedump', '--write-resource-dump', '-d', action='store_true',
-                   help="write a Resource Dump. Specify output file with --outfile and use other "
-                        "options as for --write-resourcelist")
-    loc.add_option('--write-changedump', '--write-change-dump', action='store_true',
-                   help="write a Resource Dump. Specify output file with --outfile and use other "
-                        "options as for --write-changelist")
-
-    # Specification of map between remote URI and local file paths, and remote
-    # sitemap
-    nam = p.add_option_group('FILE/URI NAMING OPTIONS')
-    nam.add_option('--outfile', type=str, action='store',
-                   help="write output to specified file rather than STDOUT or default")
-    nam.add_option('--paths', type=str, action='store',
-                   help="explicit set of paths for disk scan --resourceslist or --changelist "
-                        "generation")
-    nam.add_option('--sitemap', type=str, action='store',
-                   help="explicitly set sitemap name, overriding default sitemap.xml "
-                        "appended to first source URI specified in the mappings")
-    nam.add_option('--capabilitylist', '--capability-list', type=str, action='store',
-                   help="explicitly set capability list URI to search for instead of "
-                        "looking for the source description")
-    nam.add_option('--reference', type=str, action='store',
-                   help="reference sitemap name for --write-changelist calculation")
-    nam.add_option('--newreference', type=str, action='store',
-                   help="updated reference sitemap name for --write-changelist calculation")
-    nam.add_option('--changelist-uri', '--change-list-uri', type=str, action='store',
-                   help="explicitly set the changelist URI that will be use in --inc mode, "
-                        "overrides process of getting this from the sitemap")
-
-    lks = p.add_option_group("LINK GENERATION")
-    lks.add_option('--link', type=str, action='append',
-                   help="add discovery links to the output sitemap, "
-                        "format: rel,href[,att1=val1,att2=val2] "
-                        "(repeat option for multiple links)")
-    lks.add_option('--describedby-link', type=str, action='store',
-                   help="add an <rs:md rel=\"describedby\" link to "
-                        "a description of the feed at the URI given")
-    lks.add_option('--sourcedescription-link', '--source-description-link',
-                   type=str, action='store',
-                   help="for a Capability List add a <rs:md rel=\"up\" link to the"
-                        "Source Description document at the URI given, else ignored")
-    lks.add_option('--capabilitylist-link', '--capability-list-link',
-                   type=str, action='store',
-                   help="for all documents except a Capability List or a "
-                        "Source Description, add an <rs:md rel=\"up\" link "
-                        "to the Capability List at the URI given")
-
-    # Options that apply to multiple modes
-    opt = p.add_option_group('MISCELANEOUS OPTIONS')
-    opt.add_option('--hash', type=str, action='append',
-                   help="use specified hash types in addition to last modification time "
-                        "and size (repeatable, may include `md5`, `sha-1` and `sha-256`)")
-    opt.add_option('--checksum', action='store_true',
-                   help="use md5 checksum in addition to last modification time and size "
-                        "(same as --hash=md5)")
-    opt.add_option('--delete', action='store_true',
-                   help="allow files on destination to be deleted")
-    opt.add_option('--from', type=str, action='store', dest='from_datetime', metavar="DATETIME",
-                   help="explicit datetime value used to filter updates in change list for "
-                        "--incremental sync")
-    opt.add_option('--exclude', type=str, action='append',
-                   help="exclude resources with URI or filename matching the python regex "
-                        "supplied (see: <https://docs.python.org/2/howto/regex.html> for regex "
-                        "information, repeat option for multiple excludes)")
-    opt.add_option('--empty', action='store_true',
-                   help="combine with --changelist to write and empty changelist, perhaps with links")
-    opt.add_option('--multifile', '-m', action='store_true',
-                   help="disable reading and output of sitemapindex for multifile sitemap")
-    opt.add_option('--noauth', action='store_true',
-                   help="disable all checking of URLs to ensure that the ResourceSync "
-                        "documents refer only to resources on the same server or sub-domains. "
-                        "Use with care.")
-    opt.add_option('--strictauth', action='store_true',
-                   help="use more strict checking of URLs to ensure that the ResourceSync "
-                        "documents refer only to resources on the same server or sub-domains, "
-                        "and on the same server to sub-paths. This is the authority model "
-                        "of Sitemaps but there are legitimate uses where these rules would "
-                        "not be followed.")
-    opt.add_option('--warc', action='store_true',
-                   help="write dumps in WARC format (instead of ZIP+Sitemap default)")
-    opt.add_option('--dryrun', '-n', action='store_true',
-                   help="don't update local resources, say what would be done")
-    opt.add_option('--ignore-failures', action='store_true',
-                   help="continue past download failures")
-    # These likely only useful for experimentation
-    opt.add_option('--max-sitemap-entries', type=int, action='store',
-                   help="override default size limits")
-    # Want these to show at the end
-    opt.add_option('--verbose', '-v', action='store_true',
-                   help="verbose, show additional informational messages")
-    opt.add_option('--logger', '-l', action='store_true',
-                   help="create detailed log of client actions (will write "
-                        "to %s unless specified with --logfile" %
-                        DEFAULT_LOGFILE)
-    opt.add_option('--logfile', type='str', action='store',
-                   help="create detailed log of client actions")
-    opt.add_option('--eval', '-e', action='store_true',
-                   help="output evaluation of source/client synchronization performance... "
-                        "be warned, this is very verbose")
-
-    (args, map) = p.parse_args()
-
-    # Implement exclusive arguments and default --baseline (support for exclusive
-    # groups in argparse is incomplete is python2.6)
-    if (not args.baseline and not args.incremental and not args.audit and
-            not args.parse and not args.write_resourcelist and not args.write_changelist and
-            not args.write_capabilitylist and not args.write_sourcedescription and
-            not args.write_resourcedump and not args.write_changedump):
-        if (len(map) == 0):
-            p.error("No arguments specified (use -h for help)")
-            return
-        else:
-            args.baseline = True
-    elif (count_true_args(args.baseline, args.incremental, args.audit, args.parse,
-                          args.write_resourcelist, args.write_changelist,
-                          args.write_capabilitylist, args.write_sourcedescription,
-                          args.write_resourcedump, args.write_changedump) > 1):
-        p.error("Only one of --baseline, --incremental, --audit, --parse, --write-resourcelist, "
-                "--write-changelist, --write-capabilitylist, --write-sourcedescription, "
-                "--write-resourcedump, --write-changedump modes allowed")
-
-    # Configure logging module and create logger instance
-    init_logging(to_file=args.logger, logfile=args.logfile, default_logfile=DEFAULT_LOGFILE,
-                 verbose=args.verbose, eval_mode=args.eval)
-
-    if (args.checksum):
-        args.hash.append('md5')
-    c = Client(hashes=args.hash,
-               verbose=args.verbose,
-               dryrun=args.dryrun)
-
-    try:
-        if (map):
-            # Mappings apply to (almost) everything
-            c.set_mappings(map)
-        if (args.sitemap):
-            c.sitemap_name = args.sitemap
-        if (args.capabilitylist):
-            c.capability_list_uri = args.capabilitylist
-        if (args.warc):
-            c.dump_format = 'warc'
-        if (args.exclude):
-            c.exclude_patterns = args.exclude
-        if (args.multifile):
-            c.allow_multifile = not args.multifile
-        if (args.noauth):
-            c.noauth = args.noauth
-        if (args.strictauth):
-            c.strictauth = args.strictauth
-        if (args.max_sitemap_entries):
-            c.max_sitemap_entries = args.max_sitemap_entries
-        if (args.ignore_failures):
-            c.ignore_failures = args.ignore_failures
-
-        # Links apply to anything that writes sitemaps
-        links = parse_links(args.link)
-        # Add specific links is appropriate cases
-        if (args.capabilitylist_link and
-                not args.write_capabilitylist and
-                not args.write_sourcedescription):
-            # rel="up" to Capability List in all but Capability List
-            # and Source Description
-            links.insert(0, {'rel': 'up', 'href': args.capabilitylist_link})
-        if (args.sourcedescription_link and args.write_capabilitylist):
-            # rel="up" to Source Description from Capability List
-            links.insert(0, {'rel': 'up', 'href': args.sourcedescription_link})
-        if (args.describedby_link):
-            links.insert(0, {'rel': 'describedby',
-                             'href': args.describedby_link})
-
-        # Finally, do something...
-        if (args.baseline or args.audit):
-            c.baseline_or_audit(allow_deletion=args.delete,
-                                audit_only=args.audit)
-        elif (args.incremental):
-            c.incremental(allow_deletion=args.delete,
-                          change_list_uri=args.changelist_uri,
-                          from_datetime=args.from_datetime)
-        elif (args.parse):
-            c.parse_document()
-        elif (args.write_resourcelist or args.write_resourcedump):
-            c.write_resource_list(paths=args.paths,
-                                  outfile=args.outfile,
-                                  links=links,
-                                  dump=args.write_resourcedump)
-        elif (args.write_changelist or args.write_changedump):
-            if (not args.reference and not args.empty):
-                p.error(
-                    "Must supply --reference sitemap for --changelist, or --empty")
-            c.write_change_list(ref_sitemap=args.reference,
-                                newref_sitemap=(args.newreference if (
-                                    args.newreference) else None),
-                                empty=args.empty,
-                                paths=args.paths,
-                                outfile=args.outfile,
-                                links=links,
-                                dump=args.write_changedump)
-        elif (args.write_capabilitylist):
-            c.write_capability_list(
-                capabilities=parse_capabilities(args.write_capabilitylist),
-                outfile=args.outfile,
-                links=links)
-        elif (args.write_sourcedescription):
-            c.write_source_description(
-                capability_lists=parse_capability_lists(
-                    args.write_sourcedescription),
-                outfile=args.outfile,
-                links=links)
-        else:
-            p.error("Unknown mode requested")
-    # Any problem we expect will come as a ClientFatalError, anything else
-    # is... an exception ;-)
-    except ClientFatalError as e:
-        sys.stderr.write("\nFatalError: " + str(e) + "\n")
-
-if __name__ == '__main__':
-    main()
--- a/bin/resync-explorer
+++ b/bin/resync-explorer
@ -1,129 +0,0 @@
-#!/usr/bin/env python
-"""
-resync-explorer: The ResourceSync explorer
-
-Copyright 2012,2013 Simeon Warner
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License
-"""
-
-import optparse
-import sys
-
-from resync import __version__
-from resync.client import ClientFatalError
-from resync.client_utils import init_logging
-from resync.explorer import Explorer
-
-DEFAULT_LOGFILE = 'resync-explorer.log'
-
-
-def main():
-
-    if (sys.version_info < (2, 6)):
-        sys.exit("This program requires python version 2.6 or later")
-
-    # Options and arguments
-    p = optparse.OptionParser(description='ResourceSync explorer',
-                              usage='usage: %prog [options] uri_path local_path  (-h for help)',
-                              version='%prog ' + __version__)
-
-    # Specification of map between remote URI and local file paths, and remote
-    # sitemap
-    nam = p.add_option_group('FILE/URI NAMING OPTIONS')
-    nam.add_option('--outfile', type=str, action='store',
-                   help="write sitemap to specified file rather than STDOUT")
-    nam.add_option('--paths', type=str, action='store',
-                   help="explicit set of paths for disk scan --resourceslist or --changelist "
-                        "generation")
-    nam.add_option('--sitemap', type=str, action='store',
-                   help="explicitly set sitemap name, overriding default sitemap.xml "
-                        "appended to first source URI specified in the mappings")
-    nam.add_option('--reference', type=str, action='store',
-                   help="reference sitemap name for --changelist calculation")
-    nam.add_option('--newreference', type=str, action='store',
-                   help="updated reference sitemap name for --changelist calculation")
-    nam.add_option('--dump', metavar='DUMPFILE', type=str, action='store',
-                   help="write dump to specified file for --resourcelist or --changelist")
-    nam.add_option('--changelist-uri', '--change-list-uri', type=str, action='store',
-                   help="explicitly set the changelist URI that will be use in --inc mode, "
-                        "overrides process of getting this from the sitemap")
-
-    # Options that apply to multiple modes
-    opt = p.add_option_group('MISCELANEOUS OPTIONS')
-    opt.add_option('--hash', type=str, action='append',
-                   help="use specified hash types in addition to last modification time "
-                        "and size (repeatable, may include `md5`, `sha-1` and `sha-256`)")
-    opt.add_option('--checksum', action='store_true',
-                   help="use md5 checksum in addition to last modification time and size "
-                        "(same as --hash=md5)")
-    opt.add_option('--from', type=str, action='store', dest='from_datetime', metavar="DATETIME",
-                   help="explicit datetime value used to filter updates in change list for "
-                        "--incremental sync")
-    opt.add_option('--exclude', type=str, action='append',
-                   help="exclude resources with URI or filename matching the python regex "
-                        "supplied (see: <https://docs.python.org/2/howto/regex.html> for regex "
-                        "information, repeat option for multiple excludes)")
-    opt.add_option('--multifile', '-m', action='store_true',
-                   help="disable reading and output of sitemapindex for multifile sitemap")
-    opt.add_option('--noauth', action='store_true',
-                   help="disable checking of URL paths to ensure that the sitemaps refer "
-                        "only to resources on the same server/sub-path etc. Use with care.")
-    # These likely only useful for experimentation
-    opt.add_option('--max-sitemap-entries', type=int, action='store',
-                   help="override default size limits")
-    # Want these to show at the end
-    opt.add_option('--verbose', '-v', action='store_true',
-                   help="verbose")
-    opt.add_option('--logger', '-l', action='store_true',
-                   help="create detailed log of explorer actions (will write "
-                        "to %s unless specified with --logfile" %
-                        DEFAULT_LOGFILE)
-    opt.add_option('--logfile', type='str', action='store',
-                   help="create detailed log of client actions")
-
-    (args, map) = p.parse_args()
-
-    init_logging(to_file=args.logger, logfile=args.logfile, default_logfile=DEFAULT_LOGFILE,
-                 verbose=args.verbose)
-
-    print("----- ResourceSync Explorer -----")
-    if (args.checksum):
-        args.hash.append('md5')
-    c = Explorer(hashes=args.hash,
-                 verbose=args.verbose)
-
-    try:
-        if (map):
-            # Mappings apply to (almost) everything
-            c.set_mappings(map)
-        if (args.sitemap):
-            c.sitemap_name = args.sitemap
-        if (args.exclude):
-            c.exclude_patterns = args.exclude
-        if (args.multifile):
-            c.allow_multifile = not args.multifile
-        if (args.noauth):
-            c.noauth = args.noauth
-        if (args.max_sitemap_entries):
-            c.max_sitemap_entries = args.max_sitemap_entries
-
-        c.explore()
-
-    # Any problem we expect will come as a ClientFatalError, anything else
-    # is... an exception ;-)
-    except ClientFatalError as e:
-        sys.stderr.write("\nFatalError: " + str(e) + "\n")
-
-if __name__ == '__main__':
-    main()
--- a/docs/documentation.txt
+++ b/docs/documentation.txt
--- a/docs/examples/01_make_resourcelist.py
+++ b/docs/examples/01_make_resourcelist.py
@ -1,8 +0,0 @@
-from resync.resource_list import ResourceList
-from resync.resource import Resource
-from resync.sitemap import Sitemap
-
-rl = ResourceList()
-rl.add( Resource('http://example.com/res1', lastmod='2013-01-01') )
-rl.add( Resource('http://example.com/res2', lastmod='2013-01-02') )
-print rl.as_xml(pretty_xml=True)
--- a/docs/examples/99_make_spec_examples.pl
+++ b/docs/examples/99_make_spec_examples.pl
@ -1,190 +0,0 @@
-from resync.resource_list import ResourceList
-from resync.change_list import ChangeList
-from resync.resource_dump import ResourceDump
-from resync.resource_dump_manifest import ResourceDumpManifest
-from resync.capability_list import CapabilityList
-from resync.resource import Resource
-from resync.sitemap import Sitemap
-
-def head(txt):
-    print "\n=== "+txt+" ===\n"
-
-print "\nExample script to build objects to write the examples in the spec\n"
-
-head("Example 2.1: A Resource List")
-rl = ResourceList()
-rl.modified='2013-01-03T09:00:00Z'
-rl.add( Resource('http://example.com/res1') )
-rl.add( Resource('http://example.com/res2') )
-print rl.as_xml(pretty_xml=True)
-
-head("Example 2.2: A Resource List with additional information")
-rl = ResourceList()
-rl.modified='2013-01-03T09:00:00Z'
-rl.add( Resource('http://example.com/res1',
-	         lastmod='2013-01-02T13:00:00Z', 
-		 md5='1584abdf8ebdc9802ac0c6a7402c03b') )
-rl.add( Resource('http://example.com/res2',
-                 lastmod='2013-01-02T14:00:00Z',
-		 md5='1e0d5cb8ef6ba40c99b14c0237be735e') )
-print rl.as_xml(pretty_xml=True)
-
-head("Example 2.3: A Change List")
-cl = ChangeList()
-cl.modified="2013-01-03T11:00:00Z"
-cl.add( Resource('http://example.com/res2.pdf',
-		 lastmod='2013-01-02T13:00:00Z',
-		 change='updated') )
-cl.add( Resource('http://example.com/res3.tiff',
-	 	 lastmod='2013-01-02T18:00:00Z',
-		 change='deleted') )
-print cl.as_xml(pretty_xml=True)
-
-head("Example 2.4: A Resource Dump")
-rd = ResourceDump()
-rd.modified="2013-01-03T09:00:00Z"
-rd.add( Resource('http://example.com/resourcedump.zip',
-		 lastmod='2013-01-03T09:00:00Z') )
-print rd.as_xml(pretty_xml=True)
-
-head("Example 2.5: A Resource Dump Manifest detailing the content of a ZIP file")
-rdm = ResourceDumpManifest()
-rdm.modified="2013-01-03T09:00:00Z"
-rdm.add( Resource('http://example.com/res1',
-	          lastmod='2013-01-02T13:00:00Z', 
-		  md5='1584abdf8ebdc9802ac0c6a7402c03b6',
-	          path='/resources/res1') )
-rdm.add( Resource('http://example.com/res2',
-                  lastmod='2013-01-02T14:00:00Z',
-	   	  md5='1e0d5cb8ef6ba40c99b14c0237be735e',
-	          path='/resources/res2') )
-print rdm.as_xml(pretty_xml=True)
-
-head("Example 2.6: A Capability List with the description of the ResourceSync capabilities of a Source")
-caplist=CapabilityList()
-caplist.modified="2013-01-02T14:00:00Z"
-caplist.ln.append({'href':"http://example.com/info-about-source.xml",
-                   'rel':"describedby",
-		   'type':"application/xml"})
-caplist.add( Resource('http://example.com/dataset1/resourcelist.xml',
-                      capability="resourcelist") )
-caplist.add( Resource('http://example.com/dataset1/resourcedump.xml',
-                      capability="resourcedump") )
-caplist.add( Resource('http://example.com/dataset1/changelist.xml',
-                      capability="changelist") )
-print caplist.as_xml(pretty_xml=True)
-
-head("Example 2.7: A Resource List Index expressed using the <sitemapindex> document format")
-rl = ResourceList()
-rl.modified="2013-01-03T09:00:00Z"
-rl.add( Resource("http://example.com/resourcelist-part2.xml",
-		 lastmod="2013-01-03T09:00:00Z") )
-rl.add( Resource("http://example.com/resourcelist-part1.xml",
-                 lastmod="2013-01-03T09:00:00Z") )
-print rl.index_as_xml(pretty_xml=True)
-
-head("Example 4.1: A Resource List")
-rl = ResourceList()
-rl.modified="2013-01-03T09:00:00Z"
-rl.ln.append({"rel":"resourcesync",
-              "href":"http://example.com/dataset1/capabilitylist.xml"})
-rl.add( Resource("http://example.com/res1",
-                 lastmod="2013-01-02T13:00:00Z",
-                 md5="1584abdf8ebdc9802ac0c6a7402c03b6",
-                 length="8876",
-                 type="text/html") )
-rl.add( Resource("http://example.com/res2",
-                 lastmod="2013-01-02T14:00:00Z",
-                 md5="1e0d5cb8ef6ba40c99b14c0237be735e",
-		 sha256="854f61290e2e197a11bc91063afce22e43f8ccc655237050ace766adc68dc784",
-                 length="14599",
-                 type="application/pdf") )
-print rl.as_xml(pretty_xml=True)
-
-head("Example 4.2: A Resource List Index")
-rl = ResourceList()
-rl.modified="2013-01-03T09:00:00Z"
-rl.ln.append({"rel":"resourcesync",
-              "href":"http://example.com/dataset1/capabilitylist.xml"})
-rl.add( Resource("http://example.com/resourcelist3.xml",
-                 lastmod="2013-01-03T09:00:00Z") )
-rl.add( Resource("http://example.com/resourcelist2.xml",
-                 lastmod="2013-01-03T09:00:00Z") )
-rl.add( Resource("http://example.com/resourcelist1.xml",
-                 lastmod="2013-01-03T09:00:00Z") )
-print rl.index_as_xml(pretty_xml=True)
-
-head("Example 4.3: A Resource List with a navigational link to its parent Resource List Index")
-rl = ResourceList()
-rl.modified="2013-01-03T09:00:00Z"
-rl.ln.append({"rel":"resourcesync",
-              "href":"http://example.com/dataset1/capabilitylist.xml"})
-rl.ln.append({"rel":"up",
-              "href":"http://example.com/dataset1/resourcelist-index.xml"})
-rl.add( Resource("http://example.com/res3",
-                 lastmod="2013-01-03T09:00:00Z",
-                 md5="1584abdf8ebdc9802ac0c6a7402c8753",
-                 length="4385",
-                 type="application/pdf") )
-rl.add( Resource("http://example.com/res2",
-                 lastmod="2013-01-03T09:00:00Z",
-                 md5="4556abdf8ebdc9802ac0c6a7402c9881",
-                 length="883",
-                 type="image/png") )
-print rl.as_xml(pretty_xml=True)
-
-head("Example 5.1: A Resource Dump document")
-rd = ResourceDump()
-rd.modified = "2013-01-03T09:00:00Z"
-rd.ln.append({"rel":"resourcesync",
-              "href":"http://example.com/dataset1/capabilitylist.xml"})
-rd.add( Resource("http://example.com/resourcedump-part3.zip",
-		 lastmod="2013-01-03T09:00:00Z",
-		 type="application/zip",
-		 length="4765") )
-rd.add( Resource("http://example.com/resourcedump-part2.zip",
-		 lastmod="2013-01-03T09:00:00Z",
-                 type="application/zip",
-		 length="9875") )
-rd.add( Resource("http://example.com/resourcedump-part1.zip",
-                 lastmod="2013-01-03T09:00:00Z",
-                 type="application/zip",
-                 length="2298") )
-print rd.as_xml(pretty_xml=True)
-
-head("Example 5.2: A Resource Dump Manifest")
-rdm = ResourceDumpManifest()
-rdm.modified="2013-01-03T09:00:00Z"
-rdm.ln.append({"rel":"resourcesync",
-               "href":"http://example.com/dataset1/capabilitylist.xml"})
-rdm.add( Resource("http://example.com/res1",
-		  lastmod="2013-01-02T13:00:00Z",
-		  md5="1584abdf8ebdc9802ac0c6a7402c03b6",
-		  length="8876",
-		  type="text/html",
-		  path="/resources/res1") )
-rdm.add( Resource("http://example.com/res2",
-		  lastmod="2013-01-02T14:00:00Z",
-                  md5="1e0d5cb8ef6ba40c99b14c0237be735e",
-                  sha256="854f61290e2e197a11bc91063afce22e43f8ccc655237050ace766adc68dc784",
-		  length="14599",
-		  type="application/pdf",
-		  path="/resources/res2") )
-print rdm.as_xml(pretty_xml=True)
-
-head("Example 5.3: A Resource Dump Manifest Index")
-rdm = ResourceDumpManifest()
-rdm.modified="2013-01-03T09:00:00Z"
-rdm.ln.append({"rel":"resourcesync",
-               "href":"http://example.com/dataset1/capabilitylist.xml"})
-rdm.add( Resource("/manifests/part3.xml",
-		  lastmod="2013-01-03T09:00:00Z",
-                  path="/manifests/part3.xml") )
-rdm.add( Resource("/manifests/part2.xml",
-		  lastmod="2013-01-03T09:00:00Z",
-		  path="/manifests/part2.xml") )
-rdm.add( Resource("/manifests/part1.xml",
-		  lastmod="2013-01-03T09:00:00Z",
-		  path="/manifests/part1.xml") )
-print rdm.index_as_xml(pretty_xml=True)
-
--- a/docs/pypi_upload.md
+++ b/docs/pypi_upload.md
@ -1,56 +1,79 @@
-=======================
-Updating resync on pypi
-=======================
+# Updating resync on pypi

 Notes to remind @zimeon...

 resync is at <https://pypi.python.org/pypi/resync> on pypi

-Putting up a new version
------------------------
+## Putting up a new version

-0. In dev branch: bump version number in resync/_version.py and check CHANGES.md is up to date
-1. Check all tests good with py3 and py2 (python setup.py test)
+0. In `develop` branch: bump version number in `resync/__init__.py` and check `CHANGES.md` is up to date
+1. Check all tests good with appropriate Python 3.x (`python setup.py test` and CI)
 2. Check code is up-to-date with github version
-3. Check out master and merge in dev
-4. Check all tests good (python setup.py test)
-5. Make sure master README has correct travis-ci icon link
-6. Check branches as expected (git branch -a)
-7. Check local build and version reported OK (python setup.py build; python setup.py install; resync --version)
+3. Check out `main` and merge in `develop`
+4. Check all tests still good (`python setup.py test` and CI)
+5. Make sure main `README.md` has correct travis-ci icon link
+6. Check branches as expected (`git branch -a`)
+7. Check local build and version reported OK (`python setup.py build; python setup.py install; resync-sync -h`)
 8. Check client works with simulator:

   ```
-   simeon@RottenApple resync>resync --delete http://resync.library.cornell.edu/sim100
-   Status: NOT IN SYNC (same=92, to create=0, to update=1, to delete=0)
-   Will GET 1 resources, and delete 0 resources
-   Status:      SYNCED (same=92, created=0, updated=1, deleted=0)
-   simeon@RottenApple resync>resync -i --delete http://resync.library.cornell.edu/sim100
-   Status:  NO CHANGES (created=0, updated=0, deleted=0)
-   simeon@RottenApple resync>resync -i --delete http://resync.library.cornell.edu/sim100
-   Status:     CHANGES (created=1, updated=0, deleted=0)
-   simeon@RottenApple resync>resync -i --delete http://resync.library.cornell.edu/sim100
-   Status:  NO CHANGES (created=0, updated=0, deleted=0)
-   simeon@RottenApple resync>resync -a http://resync.library.cornell.edu/sim100
-   Status:     IN SYNC (same=94, to create=0, to update=0, to delete=0)
-   simeon@RottenApple resync>resync -a http://resync.library.cornell.edu/sim100
-   Status: NOT IN SYNC (same=94, to create=1, to update=0, to delete=0)
-   simeon@RottenApple resync>resync -i --delete http://resync.library.cornell.edu/sim100
-   Status:     CHANGES (created=1, updated=0, deleted=0)
-   simeon@RottenApple resync>resync -a http://resync.library.cornell.edu/sim100
-   Status:     IN SYNC (same=95, to create=0, to update=0, to delete=0)
-    ```
+   # Run simulator in one window
+   resync-simulator> ./resync-simulator
+
+   # Run client in another
+   resync> rm -rf localhost_8888
+
+   resync> ./resync-sync --baseline --delete http://localhost:8888/
+   Using URI mapping: http://localhost:8888/ -> localhost_8888
+   Status:     NOT IN SYNC (same=0, to create=1003, to update=0, to delete=0)
+   Will GET 1003 resources, and delete 0 resources
+   Status:          SYNCED (same=0, created=1003, updated=0, deleted=0)
+
+   resync> ./resync-sync --baseline --delete http://localhost:8888/
+   Using URI mapping: http://localhost:8888/ -> localhost_8888
+   Status:     NOT IN SYNC (same=1000, to create=2, to update=1, to delete=2)
+   Will GET 3 resources, and delete 2 resources
+   Status:          SYNCED (same=1000, created=2, updated=1, deleted=2)
+
+   resync> ./resync-sync --incremental --delete http://localhost:8888/
+   Using URI mapping: http://localhost:8888/ -> localhost_8888
+   Status: CHANGES APPLIED (created=4, updated=5, deleted=4)
+   Will apply 13 changes, and delete 4 resources
+   Status:      NO CHANGES (created=4, updated=5, deleted=4)
+
+   esync> ./resync-sync --incremental --delete http://localhost:8888/
+   Using URI mapping: http://localhost:8888/ -> localhost_8888
+   Status: CHANGES APPLIED (created=0, updated=3, deleted=0)
+   Will apply 3 changes, and delete 0 resources
+   Status:      NO CHANGES (created=0, updated=3, deleted=0)
+
+   resync> ./resync-sync --incremental --delete http://localhost:8888/; ./resync-sync --audit http://localhost:8888/
+   Using URI mapping: http://localhost:8888/ -> localhost_8888
+   Status: CHANGES APPLIED (created=1, updated=2, deleted=0)
+   Will apply 3 changes, and delete 0 resources
+   Status:      NO CHANGES (created=1, updated=2, deleted=0)
+   Using URI mapping: http://localhost:8888/ -> localhost_8888
+   Status:         IN SYNC (same=1001, to create=0, to update=0, to delete=0)
+   ```

 9. If all checks out OK, tag and push the new version to github:

-    ```
-    git tag -n1
-    #...current tags
-    git tag -a -m "ResourceSync v0.9 specification, add --paths" v0.9.3
-    git push --tags
+```
+git tag -n1
+#...current tags
+git tag -a -m "ResourceSync library and client" v2.0.0
+git push --tags
+```

-    python setup.py sdist upload
-    ```
+10. Upload to PyPI

-10. Then check on PyPI at https://pypi.python.org/pypi/resync
-11. Finally, back on dev branch start new version number by editing resync/_version.py and CHANGES.md

+```
+rm -r dist
+python setup.py sdist bdist_wheel; ls dist
+# Should be source and wheel files for just this version
+twine upload dist/*
+```
+
+10. Check on PyPI at <https://pypi.python.org/pypi/resync>
+11. Finally, back on `develop` branch start new version number by editing `resync/__init__.py` and `CHANGES.md`
--- a/docs/tutorial.md
+++ b/docs/tutorial.md
@ -1,45 +1,47 @@
-============================================
-ResourceSync tutorial exercises using resync
-============================================

+# ResourceSync tutorial exercises using resync

-Writing a Resource Dump
-----------------------
+## Writing a Resource Dump

 Relative to the base directory of the github distribution there is a directory
-`resync/test/testdata/dir1` with two files, `file_a` and `file_b`. We can create 
+`tests/testdata/dir1` with two files, `file_a` and `file_b`. We can create
 a Resource Dump of this directory, as if it were mapped to `http://example.com/dir1`,
 with the following:

 ```
-simeon@RottenApple resync>resync --resourcedump --outfile /tmp/rd.zip http://example.com/dir1 resync/test/testdata/dir1
+resync> ./resync-build -v --write-resourcedump --outfile /tmp/rd http://example.com/dir1 tests/testdata/dir1
+Scanning disk from tests/testdata/dir1
+Writing resource dump to True...
 Total size of files to include in dump 65 bytes
-Wrote ZIP file dump /tmp/rd.zip with size 595 bytes
-
-simeon@RottenApple resync>zipinfo /tmp/rd.zip 
-Archive:  /tmp/rd.zip
-Zip file size: 595 bytes, number of entries: 3
-?rw-------  2.0 unx      409 b- defN 13-May-28 18:40 manifest.xml
+Writing sitemap /tmp/rd00000.xml...
+Wrote sitemap /tmp/rd00000.xml
+Wrote ZIP file dump /tmp/rd00000.zip with size 623 bytes
+Wrote 1 dump files
+resync> zipinfo /tmp/rd00000.zip
+Archive:  /tmp/rd00000.zip
+Zip file size: 623 bytes, number of entries: 3
+?rw-------  2.0 unx      453 b- defN 20-Dec-16 10:58 manifest.xml
 -rw-r--r--  2.0 unx       20 b- defN 12-Jul-25 13:13 file_a
 -rw-r--r--  2.0 unx       45 b- defN 01-Sep-08 21:46 file_b
-3 files, 474 bytes uncompressed, 297 bytes compressed:  37.3%
+3 files, 518 bytes uncompressed, 325 bytes compressed:  37.3%
 ```

 This can then be unpacked and the files checked:

 ```
-simeon@RottenApple resync>mkdir /tmp/rd
-simeon@RottenApple resync>cd /tmp/rd
-simeon@RottenApple rd>unzip ../rd.zip 
-Archive:  ../rd.zip
+resync> mkdir /tmp/rd
+mkdir: /tmp/rd: File exists
+resync> cd /tmp/rd
+rd> unzip ../rd00000.zip
+Archive:  ../rd00000.zip
  inflating: manifest.xml            
  inflating: file_a                  
  inflating: file_b                  
-simeon@RottenApple rd>more manifest.xml 
-<?xml version=`1.0` encoding=`UTF-8`?>
-<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/"><url><loc>http://example.com/dir1/file_a</loc><lastmod>2012-07-25T17:13:46Z</lastmod><rs:md length="20" path="file_a" /></url><url><loc>http://example.com/dir1/file_b</loc><lastmod>2001-09-09T01:46:40Z</lastmod><rs:md length="45" path="file_b" /></url></urlset>
-simeon@RottenApple rd>more file_a 
+rd> more manifest.xml
+<?xml version='1.0' encoding='UTF-8'?>
+<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/"><rs:md capability="resourcedump-manifest" /><url><loc>http://example.com/dir1/file_a</loc><lastmod>2012-07-25T17:13:46Z</lastmod><rs:md length="20" path="file_a" /></url><url><loc>http://example.com/dir1/file_b</loc><lastmod>2001-09-09T01:46:40Z</lastmod><rs:md length="45" path="file_b" /></url></urlset>
+rd> more file_a
 I am file a in dir1
-simeon@RottenApple rd>more file_b 
+rd> more file_b
 I am file b in dir1, I am bigger than file_a
 ```
--- a/examples/README_client_library_in_destination.py
+++ b/examples/README_client_library_in_destination.py
@ -1,11 +1,8 @@
 #!/usr/bin/env python
-if (True): #keep indentation of README
-
-    from resync import CapabilityList
-
-    # Read Capability List and show supported capabilities
-    cl = CapabilityList()
-    cl.read("https://raw.github.com/resync/resync/0.6/resync/test/testdata/examples_from_spec/resourcesync_ex_2_6.xml")
-    for resource in cl:
-        print "supports %s (at %s)" % (resource.capability,resource.uri)
+from resync import CapabilityList

+# Read Capability List and show supported capabilities
+cl = CapabilityList()
+cl.read("https://raw.github.com/resync/resync/0.6/resync/test/testdata/examples_from_spec/resourcesync_ex_2_6.xml")
+for resource in cl:
+    print "supports %s (at %s)" % (resource.capability,resource.uri)
--- a/examples/README_client_library_in_source.py
+++ b/examples/README_client_library_in_source.py
@ -1,9 +1,7 @@
 #!/usr/bin/env python
-if (True): #keep indentation of README
+from resync import Resource, ResourceList

-    from resync import Resource,ResourceList
-
-    rl = ResourceList()
-    rl.add( Resource('http://example.com/res1', lastmod='2013-01-01') )
-    rl.add( Resource('http://example.com/res2', lastmod='2013-01-02') )
-    print rl.as_xml()
+rl = ResourceList()
+rl.add( Resource('http://example.com/res1', lastmod='2013-01-01') )
+rl.add( Resource('http://example.com/res2', lastmod='2013-01-02') )
+print rl.as_xml()
--- a/201
+++ b/201
@ -0,0 +1,201 @@
+#!/usr/bin/env python
+"""resync-build: The ResourceSync command line list builder.
+
+Copyright 2012-2020 Simeon Warner
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License
+"""
+
+import argparse
+import sys
+
+from resync import __version__
+from resync.client import Client, ClientFatalError
+from resync.client_utils import init_logging, count_true_args, parse_links, parse_capabilities, parse_capability_lists, add_shared_misc_options, process_shared_misc_options
+
+DEFAULT_LOGFILE = 'resync-client.log'
+
+
+def main():
+    """Main function to implement command line script."""
+    if (sys.version_info < (3, 5)):
+        sys.exit("This program requires python version 3.5 or later")
+
+    # Options and arguments
+    parser = argparse.ArgumentParser(
+        description="ResourceSync build script (v" + __version__ + ")",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+
+    parser._optionals = parser.add_argument_group(
+        'MODES OF OPERATION (must specify one only). This script operates only '
+        'to create ResourceSync descriptions on the local filesystem based on '
+        'local content')
+    rem = parser.add_mutually_exclusive_group(required=True)
+    rem.add_argument('--write-resourcelist', '--write-resource-list', action='store_true',
+                     help="write a resource list based on files on disk using uri=path mappings "
+                          "in reverse to calculate URIs from the local paths. Scans local disk "
+                          "based either on explicit --paths setting, else starting from all local "
+                          "paths specified in the mappings. Writes to STDOUT by default, override "
+                          "with --outfile")
+    rem.add_argument('--write-changelist', '--write-change-list', action='store_true',
+                     help="write a change list based on comparison of a reference sitemap "
+                          "(specify file with --reference) and either files on disk (using "
+                          "the mapping provided) or a second sitemap (specify file with "
+                          "--newreference). Otherwise follows --write-resourcelist options. Also accepts "
+                          "the --empty option (with no mapping) to write and empty changelist.")
+    rem.add_argument('--write-capabilitylist', '--write-capability-list', type=str, action='store',
+                     help="write a capability list based on the set of capabilities and "
+                          "URIs supplied in cap_name=URI,cap_name=URI format. Otherwise "
+                          "follows --write-resourcelist options.")
+    rem.add_argument('--write-sourcedescription', '--write-source-description', type=str, action='store',
+                     help="write a Source Description document based on the set of capability "
+                          "list URIs supplied as a comma separated list. Otherwise "
+                          "follows --write-resourcelist options.")
+    rem.add_argument('--write-resourcedump', '--write-resource-dump', '-d', action='store_true',
+                     help="write a Resource Dump. Specify output file with --outfile and use other "
+                          "options as for --write-resourcelist")
+    rem.add_argument('--write-changedump', '--write-change-dump', action='store_true',
+                     help="write a Resource Dump. Specify output file with --outfile and use other "
+                          "options as for --write-changelist")
+
+    # Positional arguments
+    map = parser.add_argument_group('URI MAPPING TO FILESYSTEM for REMOTE modes')
+    map.add_argument(metavar='uri=path | uri path', dest='map', type=str, nargs='*',
+                     help="remote URI of source for remote synchronization operations (may "
+                          "also combine uri=local path)")
+
+    nam = parser.add_argument_group('FILE/URI NAMING OPTIONS')
+    nam.add_argument('--outfile', type=str, action='store',
+                     help="write output to specified file rather than STDOUT or default")
+    nam.add_argument('--paths', type=str, action='store',
+                     help="explicit set of paths for disk scan --resourceslist or --changelist "
+                          "generation")
+    nam.add_argument('--reference', type=str, action='store',
+                     help="reference sitemap name for --write-changelist calculation")
+    nam.add_argument('--newreference', type=str, action='store',
+                     help="updated reference sitemap name for --write-changelist calculation")
+
+    lks = parser.add_argument_group("LINK GENERATION")
+    lks.add_argument('--link', type=str, action='append',
+                     help="add discovery links to the output sitemap, "
+                          "format: rel,href[,att1=val1,att2=val2] "
+                          "(repeat option for multiple links)")
+    lks.add_argument('--describedby-link', type=str, action='store',
+                     help="add an <rs:md rel=\"describedby\" link to "
+                          "a description of the feed at the URI given")
+    lks.add_argument('--sourcedescription-link', '--source-description-link',
+                     type=str, action='store',
+                     help="for a Capability List add a <rs:md rel=\"up\" link to the"
+                          "Source Description document at the URI given, else ignored")
+    lks.add_argument('--capabilitylist-link', '--capability-list-link',
+                     type=str, action='store',
+                     help="for all documents except a Capability List or a "
+                          "Source Description, add an <rs:md rel=\"up\" link "
+                          "to the Capability List at the URI given")
+
+    # Options that apply to multiple modes
+    opt = parser.add_argument_group('MISCELANEOUS OPTIONS')
+    add_shared_misc_options(opt, default_logfile=DEFAULT_LOGFILE)
+    opt.add_argument('--empty', action='store_true',
+                     help="combine with --changelist to write and empty changelist, perhaps with links")
+    opt.add_argument('--warc', action='store_true',
+                     help="write dumps in WARC format (instead of ZIP+Sitemap default)")
+    opt.add_argument('--dryrun', '-n', action='store_true',
+                     help="don't update local resources, say what would be done")
+    # These likely only useful for experimentation
+    opt.add_argument('--max-sitemap-entries', type=int, action='store',
+                     help="override default size limits")
+    opt.add_argument('--eval', '-e', action='store_true',
+                     help="output evaluation of source/client synchronization performance... "
+                          "be warned, this is very verbose")
+
+    args = parser.parse_args()
+
+    # Configure logging module and create logger instance
+    init_logging(to_file=args.logger, logfile=args.logfile, default_logfile=DEFAULT_LOGFILE,
+                 verbose=args.verbose, eval_mode=args.eval)
+
+    process_shared_misc_options(args)
+
+    c = Client(spec_version=args.spec_version,
+               hashes=args.hash,
+               verbose=args.verbose,
+               dryrun=args.dryrun)
+
+    try:
+        if (args.map):
+            # Mappings apply to (almost) everything
+            c.set_mappings(args.map)
+        if (args.warc):
+            c.dump_format = 'warc'
+        if (args.exclude):
+            c.exclude_patterns = args.exclude
+        if (args.multifile):
+            c.allow_multifile = not args.multifile
+        if (args.max_sitemap_entries):
+            c.max_sitemap_entries = args.max_sitemap_entries
+
+        # Links apply to anything that writes sitemaps
+        links = parse_links(args.link)
+        # Add specific links is appropriate cases
+        if (args.capabilitylist_link
+                and not args.write_capabilitylist
+                and not args.write_sourcedescription):
+            # rel="up" to Capability List in all but Capability List
+            # and Source Description
+            links.insert(0, {'rel': 'up', 'href': args.capabilitylist_link})
+        if (args.sourcedescription_link and args.write_capabilitylist):
+            # rel="up" to Source Description from Capability List
+            links.insert(0, {'rel': 'up', 'href': args.sourcedescription_link})
+        if (args.describedby_link):
+            links.insert(0, {'rel': 'describedby',
+                             'href': args.describedby_link})
+
+        # Finally, do something...
+        if (args.write_resourcelist or args.write_resourcedump):
+            c.write_resource_list(paths=args.paths,
+                                  outfile=args.outfile,
+                                  links=links,
+                                  dump=args.write_resourcedump)
+        elif (args.write_changelist or args.write_changedump):
+            if (not args.reference and not args.empty):
+                parser.error("Must supply --reference sitemap for --changelist, or --empty")
+            c.write_change_list(ref_sitemap=args.reference,
+                                newref_sitemap=(args.newreference if (
+                                    args.newreference) else None),
+                                empty=args.empty,
+                                paths=args.paths,
+                                outfile=args.outfile,
+                                links=links,
+                                dump=args.write_changedump)
+        elif (args.write_capabilitylist):
+            c.write_capability_list(
+                capabilities=parse_capabilities(args.write_capabilitylist),
+                outfile=args.outfile,
+                links=links)
+        elif (args.write_sourcedescription):
+            c.write_source_description(
+                capability_lists=parse_capability_lists(
+                    args.write_sourcedescription),
+                outfile=args.outfile,
+                links=links)
+        else:
+            parser.error("Unknown mode requested")
+    # Any problem we expect will come as a ClientFatalError, anything else
+    # is... an exception ;-)
+    except ClientFatalError as e:
+        sys.stderr.write("\nFatalError: " + str(e) + "\n")
+
+
+if __name__ == '__main__':
+    main()
--- a/99
+++ b/99
@ -0,0 +1,99 @@
+#!/usr/bin/env python
+"""resync-explorer: The ResourceSync explorer.
+
+Copyright 2012-2020 Simeon Warner
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License
+"""
+
+import argparse
+import sys
+
+from resync import __version__
+from resync.client import ClientFatalError
+from resync.client_utils import init_logging, add_shared_misc_options, process_shared_misc_options
+from resync.explorer import Explorer
+
+DEFAULT_LOGFILE = 'resync-explorer.log'
+
+
+def main():
+    """Main function to implement command line script."""
+    if (sys.version_info < (3, 5)):
+        sys.exit("This program requires python version 3.5 or later")
+
+    # Options and arguments
+    parser = argparse.ArgumentParser(
+        description='ResourceSync explorer (v' + __version__ + ')',
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+
+    # Specification of map between remote URI and local file paths, and remote
+    # sitemap
+    nam = parser.add_argument_group('FILE/URI NAMING OPTIONS')
+    nam.add_argument('--outfile', type=str, action='store',
+                     help="write sitemap to specified file rather than STDOUT")
+    nam.add_argument('--paths', type=str, action='store',
+                     help="explicit set of paths for disk scan --resourceslist or --changelist "
+                          "generation")
+    nam.add_argument('--sitemap', type=str, action='store',
+                     help="explicitly set sitemap name, overriding default sitemap.xml "
+                          "appended to first source URI specified in the mappings")
+    nam.add_argument('--reference', type=str, action='store',
+                     help="reference sitemap name for --changelist calculation")
+    nam.add_argument('--newreference', type=str, action='store',
+                     help="updated reference sitemap name for --changelist calculation")
+    nam.add_argument('--dump', metavar='DUMPFILE', type=str, action='store',
+                     help="write dump to specified file for --resourcelist or --changelist")
+    nam.add_argument('--changelist-uri', '--change-list-uri', type=str, action='store',
+                     help="explicitly set the changelist URI that will be use in --inc mode, "
+                          "overrides process of getting this from the sitemap")
+
+    # Options that apply to multiple modes
+    opt = parser.add_argument_group('MISCELANEOUS OPTIONS')
+    opt.add_argument('--max-sitemap-entries', type=int, action='store',
+                     help="override default size limits")
+    add_shared_misc_options(opt, default_logfile=DEFAULT_LOGFILE, include_remote=True)
+
+    args = parser.parse_args()
+
+    init_logging(to_file=args.logger, logfile=args.logfile, default_logfile=DEFAULT_LOGFILE,
+                 verbose=args.verbose)
+
+    print("----- ResourceSync Explorer -----")
+    process_shared_misc_options(args, include_remote=True)
+
+    c = Explorer(hashes=args.hash,
+                 verbose=args.verbose)
+
+    try:
+        if (args.sitemap):
+            c.sitemap_name = args.sitemap
+        if (args.exclude):
+            c.exclude_patterns = args.exclude
+        if (args.multifile):
+            c.allow_multifile = not args.multifile
+        if (args.noauth):
+            c.noauth = args.noauth
+        if (args.max_sitemap_entries):
+            c.max_sitemap_entries = args.max_sitemap_entries
+
+        c.explore()
+
+    # Any problem we expect will come as a ClientFatalError, anything else
+    # is... an exception ;-)
+    except ClientFatalError as e:
+        sys.stderr.write("\nFatalError: " + str(e) + "\n")
+
+
+if __name__ == '__main__':
+    main()
--- a/163
+++ b/163
@ -0,0 +1,163 @@
+#!/usr/bin/env python
+"""resync-sync: The ResourceSync command line synchronization client.
+
+Copyright 2012-2020 Simeon Warner
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License
+"""
+
+import argparse
+import sys
+
+from resync import __version__
+from resync.client import Client, ClientFatalError
+from resync.client_utils import init_logging, count_true_args, parse_links, parse_capabilities, parse_capability_lists, add_shared_misc_options, process_shared_misc_options
+
+DEFAULT_LOGFILE = 'resync-client.log'
+
+
+def main():
+    """Main function to implement command line script."""
+    if (sys.version_info < (3, 5)):
+        sys.exit("This program requires python version 3.5 or later")
+
+    # Options and arguments
+    parser = argparse.ArgumentParser(
+        description="ResourceSync command line client (v" + __version__ + ")",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+
+    parser._optionals = parser.add_argument_group(
+        'MODES OF OPERATION (must specify one only). A source that is specified '
+        'either in a set of uri=path mappings or else using an explicit --sitemap '
+        'location')
+    rem = parser.add_mutually_exclusive_group(required=True)
+    rem.add_argument('--baseline', '-b', action='store_true',
+                     help='baseline sync of resources from remote source (src) to local filesystem (dst)')
+    rem.add_argument('--incremental', '--inc', '-i', action='store_true',
+                     help='incremental sync of resources from remote source (src) to local filesystem (dst). Uses either timestamp recorded from last baseline or incremental sync for this source, or explicit --from parameter, to determine the earlier update timestamp to act on.')
+    rem.add_argument('--audit', '-a', action='store_true',
+                     help="audit sync state of destination wrt source")
+    rem.add_argument('--parse', '-p', action='store_true',
+                     help="parse a remote sitemap/sitemapindex (from mapping or explicit --sitemap) and show summary information including document type and number of entries")
+
+    # Positional arguments
+    map = parser.add_argument_group('URI MAPPING TO FILESYSTEM for REMOTE modes')
+    map.add_argument(metavar='uri=path | uri path', dest='map', type=str, nargs='*',
+                     help="remote URI of source for remote synchronization operations (may "
+                          "also combine uri=local path)")
+
+    # Specification of map between remote URI and local file paths, and remote
+    # sitemap
+    nam = parser.add_argument_group('FILE/URI NAMING OPTIONS')
+    nam.add_argument('--sitemap', type=str, action='store',
+                     help="explicitly set sitemap name, overriding default sitemap.xml "
+                          "appended to first source URI specified in the mappings")
+    nam.add_argument('--capabilitylist', '--capability-list', type=str, action='store',
+                     help="explicitly set capability list URI to search for instead of "
+                          "looking for the source description")
+    nam.add_argument('--reference', type=str, action='store',
+                     help="reference sitemap name for --write-changelist calculation")
+    nam.add_argument('--newreference', type=str, action='store',
+                     help="updated reference sitemap name for --write-changelist calculation")
+    nam.add_argument('--changelist-uri', '--change-list-uri', type=str, action='store',
+                     help="explicitly set the changelist URI that will be use in --inc mode, "
+                          "overrides process of getting this from the sitemap")
+
+    # Options that apply to multiple modes
+    opt = parser.add_argument_group('MISCELANEOUS OPTIONS')
+    add_shared_misc_options(opt, default_logfile=DEFAULT_LOGFILE, include_remote=True)
+    opt.add_argument('--delete', action='store_true',
+                     help="allow files on destination to be deleted")
+    opt.add_argument('--empty', action='store_true',
+                     help="combine with --changelist to write and empty changelist, perhaps with links")
+    opt.add_argument('--strictauth', action='store_true',
+                     help="use more strict checking of URLs to ensure that the ResourceSync "
+                          "documents refer only to resources on the same server or sub-domains, "
+                          "and on the same server to sub-paths. This is the authority model "
+                          "of Sitemaps but there are legitimate uses where these rules would "
+                          "not be followed.")
+    opt.add_argument('--dryrun', '-n', action='store_true',
+                     help="don't update local resources, say what would be done")
+    opt.add_argument('--ignore-failures', action='store_true',
+                     help="continue past download failures")
+    # These likely only useful for experimentation
+    opt.add_argument('--max-sitemap-entries', type=int, action='store',
+                     help="override default size limits")
+    opt.add_argument('--eval', '-e', action='store_true',
+                     help="output evaluation of source/client synchronization performance... "
+                          "be warned, this is very verbose")
+    opt.add_argument('--tries', '-t', type=int, action='store', metavar='TRIES',
+                     help="set number of tries to TRIES. The default is to retry 20 times, "
+                          "with the exception of fatal errors like \"connection refused\" "
+                          "or \"not found\" (404), which are not retried.")
+    opt.add_argument('--timeout', '-T', type=int, action='store', metavar='SECONDS',
+                     help="set the request timeout for resource downloads to SECONDS seconds")
+
+    args = parser.parse_args()
+
+    # Configure logging module and create logger instance
+    init_logging(to_file=args.logger, logfile=args.logfile, default_logfile=DEFAULT_LOGFILE,
+                 verbose=args.verbose, eval_mode=args.eval)
+
+    process_shared_misc_options(args, include_remote=True)
+
+    c = Client(spec_version=args.spec_version,
+               hashes=args.hash,
+               verbose=args.verbose,
+               dryrun=args.dryrun)
+
+    try:
+        if (args.map):
+            # Mappings apply to (almost) everything
+            c.set_mappings(args.map)
+        if (args.sitemap):
+            c.sitemap_name = args.sitemap
+        if (args.capabilitylist):
+            c.capability_list_uri = args.capabilitylist
+        if (args.exclude):
+            c.exclude_patterns = args.exclude
+        if (args.multifile):
+            c.allow_multifile = not args.multifile
+        if (args.noauth):
+            c.noauth = args.noauth
+        if (args.strictauth):
+            c.strictauth = args.strictauth
+        if (args.max_sitemap_entries):
+            c.max_sitemap_entries = args.max_sitemap_entries
+        if (args.ignore_failures):
+            c.ignore_failures = args.ignore_failures
+        if (args.tries):
+            c.tries = args.tries
+        if (args.timeout):
+            c.timeout = args.timeout
+
+        # Finally, do something...
+        if (args.baseline or args.audit):
+            c.baseline_or_audit(allow_deletion=args.delete,
+                                audit_only=args.audit)
+        elif (args.incremental):
+            c.incremental(allow_deletion=args.delete,
+                          change_list_uri=args.changelist_uri,
+                          from_datetime=args.from_datetime)
+        elif (args.parse):
+            c.parse_document()
+        else:
+            parser.error("Unknown mode requested")
+    # Any problem we expect will come as a ClientFatalError, anything else
+    # is... an exception ;-)
+    except ClientFatalError as e:
+        sys.stderr.write("\nFatalError: " + str(e) + "\n")
+
+
+if __name__ == '__main__':
+    main()
--- a/resync/init.py
+++ b/resync/init.py
@ -1,15 +1,17 @@
-"""Module config for resync."""
+"""Module config for resync.

-from resync._version import __version__
+This is the one place the version number for resync is stored.
+"""
+__version__ = '2.0.1'

 # Enable easy import for core classes, e.g.
 # from resync import Resource
-from resync.source_description import SourceDescription
-from resync.capability_list import CapabilityList
-from resync.resource_list import ResourceList
-from resync.change_list import ChangeList
-from resync.resource_dump import ResourceDump
-from resync.resource_dump_manifest import ResourceDumpManifest
-from resync.change_dump import ChangeDump
-from resync.archives import ResourceListArchive, ResourceDumpArchive, ChangeListArchive, ChangeDumpArchive
-from resync.resource import Resource
+from .source_description import SourceDescription
+from .capability_list import CapabilityList
+from .resource_list import ResourceList
+from .change_list import ChangeList
+from .resource_dump import ResourceDump
+from .resource_dump_manifest import ResourceDumpManifest
+from .change_dump import ChangeDump
+from .archives import ResourceListArchive, ResourceDumpArchive, ChangeListArchive, ChangeDumpArchive
+from .resource import Resource
--- a/resync/_version.py
+++ b/resync/_version.py
@ -1,6 +0,0 @@
-"""This is the one place the version number for resync is stored."""
-#
-# Format: x.y.z where
-# x.y is spec version, see http://www.openarchives.org/rs/x.y/
-# z is incremented for revisions within that version, 1...
-__version__ = '1.0.8'
--- a/resync/archives.py
+++ b/resync/archives.py
@ -9,10 +9,6 @@ Resource Dump Archive, and Change Dump Archive.
 """

 import collections
-try:  # python3
-    from urllib.request import URLopener
-except ImportError:  # python2
-    from urllib import URLopener

 from .list_base_with_index import ListBaseWithIndex
 from .resource import Resource
@ -23,42 +19,54 @@ class ResourceListArchive(ListBaseWithIndex):
    """Class representing an Resource List Archive."""

    def __init__(self, resources=None, md=None, ln=None, uri=None,
+                 spec_version='1.1', add_lastmod=False,
                 resources_class=None):
        """Initialize ResourceListArchive."""
        self.resources_class = list if resources_class is None else resources_class
        if (resources is None):
            resources = self.resources_class()
-        super(ResourceListArchive, self).__init__(resources=resources, md=md, ln=ln, uri=uri,
-                                                  capability_name='resourcelist-archive')
+        super(ResourceListArchive, self).__init__(
+            resources=resources, md=md, ln=ln, uri=uri,
+            capability_name='resourcelist-archive',
+            spec_version=spec_version, add_lastmod=add_lastmod)


 class ChangeListArchive(ListBaseWithIndex):
    """Class representing an Change List Archive."""

    def __init__(self, resources=None, md=None, ln=None, uri=None,
+                 spec_version='1.1', add_lastmod=False,
                 resources_class=None):
        """Initialize ChangeListArchive."""
-        super(ChangeListArchive, self).__init__(resources=resources, md=md, ln=ln, uri=uri,
-                                                capability_name='changelist-archive')
+        super(ChangeListArchive, self).__init__(
+            resources=resources, md=md, ln=ln, uri=uri,
+            capability_name='changelist-archive',
+            spec_version=spec_version, add_lastmod=add_lastmod)


 class ResourceDumpArchive(ListBaseWithIndex):
    """Class representing an Resource Dump Archive."""

    def __init__(self, resources=None, md=None, ln=None, uri=None,
+                 spec_version='1.1', add_lastmod=False,
                 resources_class=None):
        """Initialize ResourceDumpArchive."""
-        super(ResourceDumpArchive, self).__init__(resources=resources, md=md, ln=ln, uri=uri,
-                                                  capability_name='resourcedump-archive',
-                                                  resources_class=resources_class)
+        super(ResourceDumpArchive, self).__init__(
+            resources=resources, md=md, ln=ln, uri=uri,
+            capability_name='resourcedump-archive',
+            spec_version=spec_version, add_lastmod=add_lastmod,
+            resources_class=resources_class)


 class ChangeDumpArchive(ListBaseWithIndex):
    """Class representing an Change Dump Archive."""

    def __init__(self, resources=None, md=None, ln=None, uri=None,
+                 spec_version='1.1', add_lastmod=False,
                 resources_class=None):
        """Initialize ChangeDumpArchive."""
-        super(ChangeDumpArchive, self).__init__(resources=resources, md=md, ln=ln, uri=uri,
-                                                capability_name='changedump-archive',
-                                                resources_class=resources_class)
+        super(ChangeDumpArchive, self).__init__(
+            resources=resources, md=md, ln=ln, uri=uri,
+            capability_name='changedump-archive',
+            spec_version=spec_version, add_lastmod=add_lastmod,
+            resources_class=resources_class)
--- a/resync/capability_list.py
+++ b/resync/capability_list.py
@ -5,7 +5,7 @@ each capability. The Capability List object may also contain metadata
 and links like other lists.
 """

-import collections
+import collections.abc

 from .resource import Resource
 from .resource_set import ResourceSet
@ -73,12 +73,16 @@ class CapabilityList(ListBase):
    section 7 and archives specification section 6.
    """

-    def __init__(self, resources=None, md=None, ln=None, uri=None):
+    def __init__(self, resources=None, md=None, ln=None, uri=None,
+                 spec_version='1.1', add_lastmod=False):
        """Initialize CapabilityList."""
        if (resources is None):
            resources = CapabilitySet()
-        super(CapabilityList, self).__init__(resources=resources, md=md, ln=ln, uri=uri,
-                                             capability_name='capabilitylist')
+        super(CapabilityList, self).__init__(
+            resources=resources, md=md, ln=ln, uri=uri,
+            capability_name='capabilitylist',
+            spec_version=spec_version,
+            add_lastmod=add_lastmod)

    def add(self, resource, replace=False):
        """Add a resource or an iterable collection of resources.
@ -88,7 +92,7 @@ class CapabilityList(ListBase):

        See add_capability() for normal method of adding capabilities.
        """
-        if isinstance(resource, collections.Iterable):
+        if isinstance(resource, collections.abc.Iterable):
            for r in resource:
                self.resources.add(r, replace)
        else:
--- a/resync/change_dump.py
+++ b/resync/change_dump.py
@ -22,12 +22,14 @@ class ChangeDump(ResourceList):
    """

    def __init__(self, resources=None, md=None, ln=None,
-                 uri=None, allow_multifile=None, mapper=None):
+                 uri=None, allow_multifile=None, mapper=None,
+                 spec_version='1.1', add_lastmod=False):
        """Initialize ChangeDump.

        Simply sets capability_name to 'changedump' when
        subclassing ResourceList.
        """
-        super(ChangeDump, self).__init__(resources=resources, md=md, ln=ln, uri=uri,
-                                         mapper=mapper)
+        super(ChangeDump, self).__init__(
+            resources=resources, md=md, ln=ln, uri=uri, mapper=mapper,
+            spec_version=spec_version, add_lastmod=add_lastmod)
        self.capability_name = 'changedump'
--- a/resync/change_dump_manifest.py
+++ b/resync/change_dump_manifest.py
@ -23,13 +23,14 @@ class ChangeDumpManifest(ChangeList):
    """

    def __init__(self, resources=None, md=None, ln=None,
-                 uri=None, allow_multifile=None, mapper=None):
+                 uri=None, allow_multifile=None, mapper=None,
+                 spec_version='1.1', add_lastmod=False):
        """Initialize ChangeDumpManifest.

        Simply sets capability_name to 'changedump-manifest' when
        subclassing ChangeList.
        """
        super(ChangeDumpManifest, self).__init__(
-            resources=resources, md=md, ln=ln,
-            uri=uri, mapper=mapper)
+            resources=resources, md=md, ln=ln, uri=uri, mapper=mapper,
+            spec_version=spec_version, add_lastmod=add_lastmod)
        self.capability_name = 'changedump-manifest'
--- a/resync/change_list.py
+++ b/resync/change_list.py
@ -15,11 +15,7 @@ ChangeList containing descriptions pertaining to that
 particular resource.
 """

-import collections
-try:  # python3
-    from urllib.request import URLopener
-except ImportError:  # python2
-    from urllib import URLopener
+import collections.abc

 from .list_base_with_index import ListBaseWithIndex
 from .resource import Resource, ChangeTypeError
@ -30,11 +26,14 @@ class ChangeList(ListBaseWithIndex):
    """Class representing an Change List."""

    def __init__(self, resources=None, md=None, ln=None, uri=None,
-                 mapper=None, resources_class=list):
+                 mapper=None, spec_version='1.1', add_lastmod=False,
+                 resources_class=list):
        """Initialize ChangeList."""
-        super(ChangeList, self).__init__(resources=resources, md=md, ln=ln, uri=uri,
-                                         capability_name='changelist', mapper=mapper,
-                                         resources_class=resources_class)
+        super(ChangeList, self).__init__(
+            resources=resources, md=md, ln=ln, uri=uri,
+            capability_name='changelist', mapper=mapper,
+            spec_version=spec_version, add_lastmod=add_lastmod,
+            resources_class=resources_class)

    def add_if_changed(self, resource):
        """Add resource if change is not None else ChangeTypeError."""
@ -49,7 +48,7 @@ class ChangeList(ListBaseWithIndex):
        Allows multiple resource_change objects for the same
        resource (ie. URI) and preserves the order of addition.
        """
-        if isinstance(resource, collections.Iterable):
+        if isinstance(resource, collections.abc.Iterable):
            for r in resource:
                self.add_if_changed(r)
        else:
@ -64,3 +63,24 @@ class ChangeList(ListBaseWithIndex):
        for resource in resources:
            rc = Resource(resource=resource, change=change)
            self.add(rc)
+
+    def prune_updates_before(self, timestamp, spec_version='1.1'):
+        """Remove all resource updates earlier than the given timestamp.
+
+        Returns the number of entries removed. Will raise an excpetion
+        if there are any entries without a datetime (1.1) or
+        timestamp (1.0).
+        """
+        n = 0
+        pruned = []
+        use_timestamp = (spec_version == '1.0')  # Else use datetime
+        for r in self.resources:
+            ts = r.timestamp if use_timestamp else r.ts_datetime
+            if (ts is None):
+                raise Exception("Entry %s has no update datetime/timestamp" % (r.uri))
+            elif (ts >= timestamp):
+                pruned.append(r)
+            else:
+                n += 1
+        self.resources = pruned
+        return(n)
--- a/resync/client.py
+++ b/resync/client.py
@ -2,18 +2,17 @@

 import sys
 try:  # python3
-    from urllib.request import urlretrieve
    from urllib.parse import urlsplit, urlunsplit, urljoin
-except ImportError:  # python2
-    from urllib import urlretrieve
-    from urlparse import urlsplit, urlunsplit, urljoin
+except ImportError:  # pragma: no cover  python2
+    from urlparse import urlsplit, urlunsplit, urljoin  # pragma: no cover
 import os.path
 import datetime
 import distutils.dir_util
 import re
 import time
 import logging
-import requests
+import shutil
+import socket

 from .resource_list_builder import ResourceListBuilder
 from .resource_list import ResourceList
@ -27,8 +26,9 @@ from .resource import Resource
 from .url_authority import UrlAuthority
 from .hashes import Hashes
 from .client_state import ClientState
-from .client_utils import ClientFatalError, ClientError, url_or_file_open
+from .client_utils import ClientFatalError, ClientError
 from .list_base_with_index import ListBaseIndexError
+from .url_or_file_open import url_or_file_open
 from .w3c_datetime import str_to_datetime, datetime_to_str


@ -42,8 +42,9 @@ class Client(object):
      debug   - very verbose for automated analysis
    """

-    def __init__(self, hashes=None, verbose=False, dryrun=False):
+    def __init__(self, spec_version='1.1', hashes=None, verbose=False, dryrun=False):
        """Initialize Client object with default parameters."""
+        self.spec_version = spec_version
        self.hashes = set(hashes) if hashes else set()
        self.verbose = verbose
        self.dryrun = dryrun
@ -62,6 +63,8 @@ class Client(object):
        self.ignore_failures = False
        self.pretty_xml = True
        self.fake_input = None
+        self.tries = 20
+        self.timeout = None
        # Default file names
        self.status_file = '.resync-client-status.cfg'
        self.default_resource_dump = 'resourcedump.zip'
@ -222,7 +225,7 @@ class Client(object):
        rlb = ResourceListBuilder(set_hashes=self.hashes, mapper=self.mapper)
        rlb.set_path = set_path
        try:
-            rlb.add_exclude_files(self.exclude_patterns)
+            rlb.add_exclude_patterns(self.exclude_patterns)
            rl = rlb.from_disk(paths=paths)
        except ValueError as e:
            raise ClientFatalError(str(e))
@ -381,12 +384,14 @@ class Client(object):
        # Check all changes have timestamp and record last
        self.last_timestamp = 0
        for resource in src_change_list:
-            if (resource.timestamp is None):
+            if resource.timestamp is None and resource.ts_datetime is None:
                raise ClientFatalError(
-                    "Aborting - missing timestamp for change in %s" %
-                    (uri))
-            if (resource.timestamp > self.last_timestamp):
-                self.last_timestamp = resource.timestamp
+                    "Aborting - no datetime or lastmod for change in %s" %
+                    (resource.uri))
+            # Work with 1.0 or 1.1 -- use datetime if given, else lastmod
+            ts = resource.ts_datetime if resource.ts_datetime is not None else resource.timestamp
+            if (ts > self.last_timestamp):
+                self.last_timestamp = ts
        # 4. Check that the change list has authority over URIs listed
        # FIXME - What does authority mean for change list? Here use both the
        # change list URI and, if we used it, the sitemap URI
@ -395,14 +400,14 @@ class Client(object):
            if (not change_list_uri):
                uauth_sm = UrlAuthority(self.sitemap)
                for resource in src_change_list:
-                    if (not uauth_cs.has_authority_over(resource.uri) and
-                            (change_list_uri or not uauth_sm.has_authority_over(resource.uri))):
+                    if (not uauth_cs.has_authority_over(resource.uri)
+                            and (change_list_uri or not uauth_sm.has_authority_over(resource.uri))):
                        raise ClientFatalError(
                            "Aborting as change list (%s) mentions resource at a location it does not have authority over (%s), override with --noauth" %
                            (change_list, resource.uri))
        # 5. Prune entries before starting timestamp and dupe changes for a
        # resource
-        num_skipped = src_change_list.prune_before(from_timestamp)
+        num_skipped = src_change_list.prune_updates_before(from_timestamp, spec_version=self.spec_version)
        if (num_skipped > 0):
            self.logger.info(
                "Skipped %d changes before %s" %
@ -498,16 +503,33 @@ class Client(object):
                (resource.uri, filename))
        else:
            # 1. GET
-            try:
-                urlretrieve(resource.uri, filename)
-                num_updated += 1
-            except IOError as e:
-                msg = "Failed to GET %s -- %s" % (resource.uri, str(e))
-                if (self.ignore_failures):
-                    self.logger.warning(msg)
-                    return(num_updated)
-                else:
-                    raise ClientFatalError(msg)
+            for try_i in range(1, self.tries + 1):
+                try:
+                    with url_or_file_open(resource.uri, timeout=self.timeout) as fh_in:
+                        with open(filename, 'wb') as fh_out:
+                            shutil.copyfileobj(fh_in, fh_out)
+                    num_updated += 1
+                    break
+                except socket.timeout as e:
+                    if try_i < self.tries:
+                        msg = 'Download timed out, retrying...'
+                        self.logger.info(msg)
+                        # Continue loop
+                    else:
+                        # No more tries left, so fail
+                        msg = "Failed to GET %s after %s tries -- %s" % (resource.uri, self.tries, str(e))
+                        if (self.ignore_failures):
+                            self.logger.warning(msg)
+                            return(num_updated)
+                        else:
+                            raise ClientFatalError(msg)
+                except IOError as e:
+                    msg = "Failed to GET %s -- %s" % (resource.uri, str(e))
+                    if (self.ignore_failures):
+                        self.logger.warning(msg)
+                        return(num_updated)
+                    else:
+                        raise ClientFatalError(msg)
            # 2. set timestamp if we have one
            if (resource.timestamp is not None):
                unixtime = int(resource.timestamp)  # no fractional
@ -566,8 +588,8 @@ class Client(object):
        """
        num_deleted = 0
        uri = resource.uri
-        if (resource.timestamp is not None and
-                resource.timestamp > self.last_timestamp):
+        if (resource.timestamp is not None
+                and resource.timestamp > self.last_timestamp):
            self.last_timestamp = resource.timestamp
        if (allow_deletion):
            if (self.dryrun):
--- a/resync/client_state.py
+++ b/resync/client_state.py
@ -14,8 +14,8 @@ import time
 import logging
 try:  # python3
    from configparser import ConfigParser, NoSectionError, NoOptionError
-except ImportError:  # python2
-    from ConfigParser import SafeConfigParser as ConfigParser, NoSectionError, NoOptionError
+except ImportError:  # pragma: no cover python2
+    from ConfigParser import SafeConfigParser as ConfigParser, NoSectionError, NoOptionError  # pragma: no cover


 class ClientState(object):
--- a/resync/client_utils.py
+++ b/resync/client_utils.py
@ -1,31 +1,14 @@
 """ResourceSync Client Utilities.

-Factor out code shared by both the resync and resync-explorer
-clients.
-
-Copyright 2012-2016 Simeon Warner
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License
+Code shared by client scripts.
 """

-try:  # python3
-    from urllib.request import urlopen
-except ImportError:  # python2
-    from urllib import urlopen
+import argparse
+from datetime import datetime
 import logging
 import logging.config
-from datetime import datetime
-import re
+
+from .url_or_file_open import set_url_or_file_open_config


 class ClientFatalError(Exception):
@ -180,8 +163,60 @@ def parse_capability_lists(cls_str):
    return(cls_str.split(','))


-def url_or_file_open(uri):
-    """Wrapper around urlopen() to prepend file: if no scheme provided."""
-    if (not re.match(r'''\w+:''', uri)):
-        uri = 'file:' + uri
-    return(urlopen(uri))
+def add_shared_misc_options(opt, default_logfile, include_remote=False):
+    """Add shared miscellaneous options to the argument_group opt.
+
+    Options that the resync-sync, resync-build and resync-explorer scripts use.
+    """
+    opt.add_argument('--hash', type=str, action='append',
+                     help="use specified hash types in addition to last modification time "
+                          "and size (repeatable, may include `md5`, `sha-1` and `sha-256`)")
+    opt.add_argument('--checksum', action='store_true',
+                     help="use md5 checksum in addition to last modification time and size "
+                          "(same as --hash=md5)")
+    opt.add_argument('--from', type=str, action='store', dest='from_datetime', metavar="DATETIME",
+                     help="explicit datetime value used to filter updates in change list for "
+                          "--incremental sync")
+    opt.add_argument('--exclude', type=str, action='append',
+                     help="exclude resources with URI or filename matching the python regex "
+                          "supplied (see: <https://docs.python.org/3/howto/regex.html> for regex "
+                          "information, repeat option for multiple excludes)")
+    opt.add_argument('--multifile', '-m', action='store_true',
+                     help="disable reading and output of sitemapindex for multifile sitemap")
+    if include_remote:
+        opt.add_argument('--noauth', action='store_true',
+                         help="disable checking of URL paths to ensure that the sitemaps refer "
+                              "only to resources on the same server/sub-path etc. Use with care.")
+        opt.add_argument('--access-token', type=str, default=None,
+                         help="include this access token (a bearer token) in web requests")
+        opt.add_argument('--delay', type=float, default=None,
+                         help="add a delay between web requests (default is None)")
+    # Want these to show at the end
+    opt.add_argument('--logger', '-l', action='store_true',
+                     help="create detailed log of client actions (will write "
+                          "to %s unless specified with --logfile" % (default_logfile))
+    opt.add_argument('--logfile', type=str, action='store',
+                     help="create detailed log of client actions")
+    opt.add_argument('--spec-version', default='1.1', choices=('1.0', '1.1'),
+                     help="follow given ResourceSync specification version. The key difference is that v1.0 "
+                          "used lastmod for the time of a change (often also the resource Last-Modification "
+                          "time but not always). In v1.1 the rs:md datetime attribute in a ChangeList "
+                          "indicates the time of the change, and use of lastmod is entirely optional")
+    opt.add_argument('--verbose', '-v', action='store_true',
+                     help="verbose, show additional informational messages")
+
+
+def process_shared_misc_options(args, include_remote=False):
+    """Process shared miscellaneous options in args.
+
+    Parse options that the resync-sync, resync-build and resync-explorer scripts use.
+    """
+    if args.checksum:
+        args.hash.append('md5')
+    if include_remote:
+        if args.access_token:
+            set_url_or_file_open_config('bearer_token', args.access_token)
+        if args.delay:
+            if args.delay < 0.0:
+                raise argparse.ArgumentTypeError("--delay must be non-negative!")
+            set_url_or_file_open_config('delay', args.delay)
--- a/resync/dump.py
+++ b/resync/dump.py
@ -96,7 +96,7 @@ class Dump(object):
        # without this installed
        try:
            from warc import WARCFile, WARCHeader, WARCRecord
-        except:
+        except ImportError:
            raise DumpError("Failed to load WARC library")
        wf = WARCFile(dumpfile, mode="w", compress=self.compress)
        # Add all files in the resources
@ -175,8 +175,7 @@ class Dump(object):
            manifest.add(resource)
            manifest_size += resource.length
            manifest_files += 1
-            if (manifest_size >= self.max_size or
-                    manifest_files >= self.max_files):
+            if (manifest_size >= self.max_size or manifest_files >= self.max_files):
                yield(manifest)
                # Need to start a new manifest
                manifest = self.manifest_class()
--- a/resync/explorer.py
+++ b/resync/explorer.py
@ -17,14 +17,14 @@ import distutils.dir_util
 import re
 import time
 import logging
-import requests

 from .mapper import Mapper
 from .sitemap import Sitemap
 from .client import Client, ClientFatalError
 from .client_state import ClientState
-from .client_utils import ClientFatalError, url_or_file_open
+from .client_utils import ClientFatalError
 from .resource import Resource
+from .url_or_file_open import url_or_file_open
 from .w3c_datetime import str_to_datetime, datetime_to_str


@ -163,7 +163,7 @@ class Explorer(Client):
                caps = 'resource'
            else:
                caps = self.allowed_entries(capability)
-        elif (r.capability is 'resource'):
+        elif (r.capability == 'resource'):
            caps = r.capability
        else:
            caps = [r.capability]
@ -277,39 +277,36 @@ class Explorer(Client):
        print("HEAD %s" % (uri))
        if (re.match(r'^\w+:', uri)):
            # Looks like a URI
-            response = requests.head(uri)
+            with url_or_file_open(uri, method='HEAD') as response:
+                status_code = response.code()
+                headers = response.headers()
        else:
            # Mock up response if we have a local file
-            response = self.head_on_file(uri)
-        print("  status: %s" % (response.status_code))
-        if (response.status_code == '200'):
+            (status_code, headers) = self.head_on_file(uri)
+        print("  status: %s" % (status_code))
+        if (status_code == '200'):
            # print some of the headers
            for header in ['content-length', 'last-modified',
                           'lastmod', 'content-type', 'etag']:
-                if header in response.headers:
+                if header in headers:
                    check_str = ''
-                    if (check_headers is not None and
-                            header in check_headers):
-                        if (response.headers[header] == check_headers[header]):
+                    if (check_headers is not None and header in check_headers):
+                        if (headers[header] == check_headers[header]):
                            check_str = ' MATCHES EXPECTED VALUE'
                        else:
                            check_str = ' EXPECTED %s' % (
                                check_headers[header])
-                    print(
-                        "  %s: %s%s" %
-                        (header, response.headers[header], check_str))
+                    print("  %s: %s%s" % (header, headers[header], check_str))

    def head_on_file(self, file):
-        """Mock up requests.head(..) response on local file."""
-        response = HeadResponse()
-        if (not os.path.isfile(file)):
-            response.status_code = '404'
-        else:
-            response.status_code = '200'
-            response.headers[
-                'last-modified'] = datetime_to_str(os.path.getmtime(file))
-            response.headers['content-length'] = os.path.getsize(file)
-        return(response)
+        """Get fake status code and headers from local file."""
+        status_code = '404'
+        headers = {}
+        if os.path.isfile(file):
+            status_code = '200'
+            headers['last-modified'] = datetime_to_str(os.path.getmtime(file))
+            headers['content-length'] = os.path.getsize(file)
+        return(status_code, headers)

    def allowed_entries(self, capability):
        """Return list of allowed entries for given capability document.
@ -367,15 +364,6 @@ class XResource(object):
        self.checks = checks


-class HeadResponse(object):
-    """Object to mock up requests.head(...) response."""
-
-    def __init__(self):
-        """Initialize with no status_code and no headers."""
-        self.status_code = None
-        self.headers = {}
-
-
 class ExplorerQuit(Exception):
    """Exception raised when user quits normally, no error."""

--- a/resync/hashes.py
+++ b/resync/hashes.py
@ -34,14 +34,16 @@ class Hashes(object):
    up [http://docs.python.org/library/base64.html]
    """

+    NAME_TO_ATTRIBUTE = {'md5': 'md5', 'sha-1': 'sha1', 'sha-256': 'sha256'}
+
    def __init__(self, hashes=None, file=None):
-        """Initialize Hasher object with types of hash to caluclate.
+        """Initialize Hashes object with types of hash to caluclate.

        If file is supplied then compute for that file.
        """
        self.hashes = set()
        for hash in hashes:
-            if (hash not in ['md5', 'sha-1', 'sha-256']):
+            if (hash not in self.NAME_TO_ATTRIBUTE.keys()):
                raise Exception("Hash type %s not supported" % (hash))
            self.hashes.add(hash)
        #
@ -74,14 +76,24 @@ class Hashes(object):
            data = f.read(block_size)
            if not data:
                break
-            if (self.md5_calc is not None):
+            if self.md5_calc is not None:
                self.md5_calc.update(data)
-            if (self.sha1_calc is not None):
+            if self.sha1_calc is not None:
                self.sha1_calc.update(data)
-            if (self.sha256_calc is not None):
+            if self.sha256_calc is not None:
                self.sha256_calc.update(data)
        f.close()

+    def set(self, resource):
+        """Set hash values for resource from current file.
+
+        Assumes that resource has appropriate attributes or setters
+        with names md5, sha1, etc. and that hashes have been calculated.
+        """
+        for hash in self.hashes:
+            att = self.NAME_TO_ATTRIBUTE[hash]
+            setattr(resource, att, getattr(self, att))
+
    @property
    def md5(self):
        """Return MD5 hash calculated."""
--- a/resync/list_base.py
+++ b/resync/list_base.py
@ -5,25 +5,16 @@ intended as the base class for ResourceList, ChangeList,
 CapabilityList etc.. Adds common read() and write() methods.
 """

-import collections
-import os
 from datetime import datetime
+import io
+import os
 import re
 import sys
-try:  # python2
-    # Must try this first as io also exists in python2
-    # but in the wrong one!
-    import StringIO as io
-except ImportError:  # python3
-    import io
 import logging
-try:  # python3
-    from urllib.request import URLopener
-except ImportError:  # python2
-    from urllib import URLopener

 from .resource_container import ResourceContainer
 from .sitemap import Sitemap
+from .url_or_file_open import url_or_file_open


 class ListBase(ResourceContainer):
@ -39,15 +30,26 @@ class ListBase(ResourceContainer):

    ln - link information for the list (<rs:ln>)

-    sitemapindex - defaults to False, set True if this is an index object
+    uri - the URL of this list
+
+    capability_name -
+
+    spec_version - default to None for latest version supported, else explicit
+        version such as '1.0'
+
+    Internal variables:
+      sitemapindex - defaults to False, set True if this is an index object
+      pretty_xml - defaults to False, set True for more human readable output
    """

    def __init__(self, resources=None, count=None, md=None, ln=None, uri=None,
-                 capability_name='unknown'):
+                 capability_name='unknown', spec_version='1.1', add_lastmod=False):
        """Initialize ListBase."""
        super(ListBase, self).__init__(resources=resources, md=md, ln=ln, uri=uri,
                                       capability_name=capability_name)
        self.count = count
+        self.spec_version = spec_version
+        self.add_lastmod = add_lastmod  # Optional in v1.1
        self.sitemapindex = False
        self.pretty_xml = False
        #
@ -96,7 +98,7 @@ class ListBase(ResourceContainer):
        """
        if (uri is not None):
            try:
-                fh = URLopener().open(uri)
+                fh = url_or_file_open(uri)
            except IOError as e:
                raise Exception(
                    "Failed to load sitemap/sitemapindex from %s (%s)" %
@ -107,7 +109,7 @@ class ListBase(ResourceContainer):
            # Legacy support for str argument, see
            # https://github.com/resync/resync/pull/21
            # One test for this in tests/test_list_base.py
-            self.logger.warn(
+            self.logger.warning(
                "Legacy parse(str=...), use parse(str_data=...) instead")
            fh = io.StringIO(kwargs['str'])
        if (fh is None):
@ -147,4 +149,6 @@ class ListBase(ResourceContainer):

    def new_sitemap(self):
        """Create new Sitemap object with default settings."""
-        return Sitemap(pretty_xml=self.pretty_xml)
+        return Sitemap(pretty_xml=self.pretty_xml,
+                       spec_version=self.spec_version,
+                       add_lastmod=self.add_lastmod)
--- a/resync/list_base_with_index.py
+++ b/resync/list_base_with_index.py
@ -11,17 +11,14 @@ from datetime import datetime
 import re
 import sys
 import itertools
-try:  # python3
-    from urllib.request import URLopener
-except ImportError:  # python2
-    from urllib import URLopener

+from .hashes import Hashes
 from .list_base import ListBase
+from .mapper import Mapper, MapperError
 from .resource import Resource
 from .sitemap import Sitemap
-from .mapper import Mapper, MapperError
 from .url_authority import UrlAuthority
-from .hashes import Hashes
+from .url_or_file_open import url_or_file_open


 class ListBaseWithIndex(ListBase):
@ -52,13 +49,20 @@ class ListBaseWithIndex(ListBase):

    def __init__(self, resources=None, count=None, md=None, ln=None, uri=None,
                 capability_name='unknown', allow_multifile=None, mapper=None,
-                 resources_class=None):
+                 spec_version='1.1', add_lastmod=False, resources_class=None):
        """Initialize ListBaseWithIndex."""
        self.resources_class = list if resources_class is None else resources_class
        if (resources is None):
            resources = self.resources_class()
-        super(ListBaseWithIndex, self).__init__(resources=resources, count=count, md=md, ln=ln,
-                                                uri=uri, capability_name=capability_name)
+        super(ListBaseWithIndex, self).__init__(
+            resources=resources,
+            count=count,
+            md=md,
+            ln=ln,
+            uri=uri,
+            capability_name=capability_name,
+            spec_version=spec_version,
+            add_lastmod=add_lastmod)
        # specific to lists with indexes
        self.max_sitemap_entries = 50000
        self.mapper = mapper
@ -84,7 +88,7 @@ class ListBaseWithIndex(ListBase):
        are mapped to the filesystem also.
        """
        try:
-            fh = URLopener().open(uri)
+            fh = url_or_file_open(uri)
            self.num_files += 1
        except IOError as e:
            raise IOError(
@ -97,7 +101,7 @@ class ListBaseWithIndex(ListBase):
            self.logger.debug(
                "Read %d bytes from %s" %
                (self.content_length, uri))
-        except KeyError:
+        except (KeyError, TypeError):
            # If we don't get a length then c'est la vie
            self.logger.debug("Read ????? bytes from %s" % (uri))
            pass
@ -150,13 +154,13 @@ class ListBaseWithIndex(ListBase):
            else:
                # The individual sitemaps should be at a URL (scheme/server/path)
                # that the sitemapindex URL can speak authoritatively about
-                if (self.check_url_authority and
-                        not UrlAuthority(sitemapindex_uri).has_authority_over(sitemap_uri)):
+                if (self.check_url_authority
+                        and not UrlAuthority(sitemapindex_uri).has_authority_over(sitemap_uri)):
                    raise ListBaseIndexError(
                        "The sitemapindex (%s) refers to sitemap at a location it does not have authority over (%s)" %
                        (sitemapindex_uri, sitemap_uri))
        try:
-            fh = URLopener().open(sitemap_uri)
+            fh = url_or_file_open(sitemap_uri)
            self.num_files += 1
        except IOError as e:
            raise ListBaseIndexError(
@ -166,7 +170,7 @@ class ListBaseWithIndex(ListBase):
        try:
            self.content_length = int(fh.info()['Content-Length'])
            self.bytes_read += self.content_length
-        except KeyError:
+        except (KeyError, TypeError):
            # If we don't get a length then c'est la vie
            pass
        self.logger.info(
@ -187,8 +191,8 @@ class ListBaseWithIndex(ListBase):
        In the case that no len() is available for self.resources then
        then self.count must be set beforehand to avoid an exception.
        """
-        if (self.max_sitemap_entries is None or
-                len(self) <= self.max_sitemap_entries):
+        if (self.max_sitemap_entries is None
+                or len(self) <= self.max_sitemap_entries):
            return(False)
        return(int(math.ceil(len(self) / float(self.max_sitemap_entries))))

@ -336,6 +340,12 @@ class ListBaseWithIndex(ListBase):
            s.resources_as_xml(index, sitemapindex=True, fh=f)
            f.close()
            self.logger.info("Wrote sitemapindex %s" % (basename))
+        elif self.sitemapindex:
+            f = open(basename, 'w')
+            self.logger.info("Writing sitemapindex %s..." % (basename))
+            s.resources_as_xml(chunk, sitemapindex=True, fh=f)
+            f.close()
+            self.logger.info("Wrote sitemapindex %s" % (basename))
        else:
            f = open(basename, 'w')
            self.logger.info("Writing sitemap %s..." % (basename))
@ -403,7 +413,7 @@ class ListBaseWithIndex(ListBase):
        Test is to see whether have either an explicit file: URI or whether
        there is no scheme name.
        """
-        return(re.match('file:', uri) or not re.match('\w{3,4}:', uri))
+        return(re.match(r'file:', uri) or not re.match(r'\w{3,4}:', uri))


 class ListBaseIndexError(Exception):
--- a/resync/mapper.py
+++ b/resync/mapper.py
@ -5,8 +5,8 @@ import os.path
 import re
 try:  # python3
    from urllib.parse import urlparse
-except ImportError:  # python2
-    from urlparse import urlparse
+except ImportError:  # pragma: no cover  python2
+    from urlparse import urlparse  # pragma: no cover
 import logging


@ -43,25 +43,25 @@ class Mapper():
        source base URI. In the case that the source base URI is a local
        path already then an indentity mapping is used.
        """
-        if (use_default_path and
-                len(mappings) == 1 and
-                re.search(r"=", mappings[0]) is None):
+        if (use_default_path
+                and len(mappings) == 1
+                and re.search(r"=", mappings[0]) is None):
            path = self.path_from_uri(mappings[0])
            self.logger.warning("Using URI mapping: %s -> %s" %
                                (mappings[0], path))
            self.mappings.append(Map(mappings[0], path))
-        elif (len(mappings) == 2 and
-              re.search(r"=", mappings[0]) is None and
-              re.search(r"=", mappings[1]) is None):
+        elif (len(mappings) == 2
+                and re.search(r"=", mappings[0]) is None
+                and re.search(r"=", mappings[1]) is None):
            self.mappings.append(Map(mappings[0], mappings[1]))
        else:
            for mapping in mappings:
-                l = mapping.split('=')
-                if (len(l) != 2):
+                entry = mapping.split('=')
+                if (len(entry) != 2):
                    raise MapperError(
                        "Bad mapping argument (%s), got %s" %
-                        (mapping, str(l)))
-                (src_uri, dst_path) = l
+                        (mapping, str(entry)))
+                (src_uri, dst_path) = entry
                # Check for dupes
                for map in self.mappings:
                    if (src_uri == map.src_uri):
@ -129,10 +129,10 @@ class Mapper():
        if (netloc == ''):
            return(uri)
        path = '/'.join([netloc, path])
-        path = re.sub('[^\w\-\.]', '_', path)
-        path = re.sub('__+', '_', path)
-        path = re.sub('[_\.]+$', '', path)
-        path = re.sub('^[_\.]+', '', path)
+        path = re.sub(r'[^\w\-\.]', '_', path)
+        path = re.sub(r'__+', '_', path)
+        path = re.sub(r'[_\.]+$', '', path)
+        path = re.sub(r'^[_\.]+', '', path)
        return(path)

    def __repr__(self):
@ -173,23 +173,35 @@ class Map:
        This does not rely on the destination filepath actually
        existing on the local filesystem, just on pattern matching.
        Return source URI on success, None on failure.
+
+        Relies upon self.dst_path and self.src_path not including trailing
+        slashes. However, a match of just self.dst_path withouth a trailing
+        slash will return self.src_path with a trailing slash.
        """
-        m = re.match(self.dst_path + "/(.*)$", dst_file)
+        m = re.match(self.dst_path + "(/.*)?$", dst_file)
        if (m is None):
            return(None)
        rel_path = m.group(1)
-        return(self.src_uri + '/' + rel_path)
+        if rel_path is None:
+            rel_path = '/'
+        return self.src_uri + rel_path

    def src_to_dst(self, src_uri):
        """Return the dst filepath from the src URI.

-        Returns None on failure, destination path on success.
+        Returns None on failure, local destination path on success.
+
+        Relies upon self.dst_path and self.src_path not including trailing
+        slashes. However, a match of just self.src_path withouth a trailing
+        slash will return self.dst_path with a trailing slash.
        """
-        m = re.match(self.src_uri + "/(.*)$", src_uri)
+        m = re.match(self.src_uri + "(/.*)?$", src_uri)
        if (m is None):
            return(None)
        rel_path = m.group(1)
-        return(self.dst_path + '/' + rel_path)
+        if rel_path is None:
+            rel_path = '/'
+        return self.dst_path + rel_path

    def unsafe(self):
        """True if the mapping is unsafe for an update.
--- a/resync/resource.py
+++ b/resync/resource.py
@ -1,11 +1,9 @@
 """ResourceSync Resources - information about a web resource and changes."""

 import re
-try:  # python3
-    from urllib.parse import urlparse
-except:  # python2
-    from urlparse import urlparse
 from posixpath import basename
+from urllib.parse import urlparse
+
 from .w3c_datetime import str_to_datetime, datetime_to_str


@ -33,12 +31,13 @@ class Resource(object):
        mime_type - MIME type
        md5, sha1, sha256 - digests, have hash accessor
        change - change type
+        ts_datetime - time of the change, has datetime accessor
        path - path in dump

    If non-core attributes are needed then the '_extra' attribute
    has a dict of values. The ones explicitly used here are:

-        capability - Capability nam
+        capability - Capability name
        ts_at - at time, has md_at accessor
        ts_completed - completed time, has md_completed accessor
        ts_from - from time, has md_from accessor
@ -55,20 +54,22 @@ class Resource(object):
    """

    __slots__ = ('uri', 'timestamp', 'length', 'mime_type',
-                 'md5', 'sha1', 'sha256', 'change', 'path',
-                 '_extra', 'ln')
+                 'md5', 'sha1', 'sha256', 'change', 'ts_datetime',
+                 'path', 'ln', '_extra')

    CHANGE_TYPES = ['created', 'updated', 'deleted']

    def __init__(self, uri=None, timestamp=None, length=None,
-                 md5=None, sha1=None, sha256=None, mime_type=None,
-                 change=None, path=None, lastmod=None,
+                 mime_type=None, md5=None, sha1=None, sha256=None,
+                 change=None, ts_datetime=None, path=None, ln=None,
+                 # the following in _extra
                 capability=None,
-                 ts_at=None, md_at=None,
-                 ts_completed=None, md_completed=None,
-                 ts_from=None, md_from=None,
-                 ts_until=None, md_until=None,
-                 resource=None, ln=None):
+                 ts_at=None, ts_completed=None, ts_from=None, ts_until=None,
+                 # and the following via setters
+                 lastmod=None, datetime=None,
+                 md_at=None, md_completed=None, md_from=None, md_until=None,
+                 # and finally another Resource to copy from
+                 resource=None):
        """Initialize Resource object.

        Initialize either from parameters specified or from an existing
@ -84,15 +85,17 @@ class Resource(object):
        self.sha1 = None
        self.sha256 = None
        self.change = None
+        self.ts_datetime = None  # Added in ResourceSync v1.1
        self.path = None
-        self._extra = None
        self.ln = None
-        # Create from a Resource-like object? Copy any relevant attributes
+        self._extra = None
+        # Create from a Resource-like object? Copy any attributes, both ones
+        # that have slots and ones that live in _extra
        if (resource is not None):
-            for att in ['uri', 'timestamp', 'length', 'md5', 'sha1', 'sha256',
-                        'change', 'path', 'capability',
-                        'ts_at', 'md_at', 'ts_completed', 'md_completed',
-                        'ts_from', 'md_from', 'ts_until', 'md_until', 'ln']:
+            for att in ['uri', 'timestamp', 'length', 'mime_type',
+                        'md5', 'sha1', 'sha256', 'change', 'ts_datetime', 'path', 'ln',
+                        # the following in _extra
+                        'capability', 'ts_at', 'ts_completed', 'ts_from', 'ts_until']:
                if hasattr(resource, att):
                    setattr(self, att, getattr(resource, att))
        # Any arguments will then override
@ -112,6 +115,8 @@ class Resource(object):
            self.mime_type = mime_type
        if (change is not None):
            self.change = change
+        if (ts_datetime is not None):
+            self.ts_datetime = ts_datetime
        if (path is not None):
            self.path = path
        if (ts_at is not None):
@ -129,6 +134,8 @@ class Resource(object):
        # Timestamp setters
        if (lastmod is not None):
            self.lastmod = lastmod
+        if (datetime is not None):
+            self.datetime = datetime
        if (md_at is not None):
            self.md_at = md_at
        if (md_completed is not None):
@ -139,7 +146,7 @@ class Resource(object):
            self.md_until = md_until
        # Sanity check
        if (self.uri is None):
-            raise ValueError("Cannot create resource without a URI")
+            raise ValueError("Cannot create Resource without a URI")

    def __setattr__(self, prop, value):
        """Attribute setter with check and support for extra attributes.
@ -149,8 +156,8 @@ class Resource(object):
        the idea of extra properties.
        """
        # Add validity check for self.change
-        if (prop == 'change' and Resource.CHANGE_TYPES and
-                value is not None and value not in Resource.CHANGE_TYPES):
+        if (prop == 'change' and Resource.CHANGE_TYPES
+                and value is not None and value not in Resource.CHANGE_TYPES):
            raise ChangeTypeError(value)
        else:
            try:
@ -174,14 +181,29 @@ class Resource(object):

    @property
    def lastmod(self):
-        """The Last-Modified data in W3C Datetime syntax, Z notation."""
+        """The Last-Modified date for the resource in W3C Datetime syntax, Z notation."""
        return datetime_to_str(self.timestamp)

    @lastmod.setter
    def lastmod(self, lastmod):
-        """Set timestamp from a W3C Datetime Last-Modified value."""
+        """Set last modified timestamp from a W3C Datetime Last-Modified value."""
        self.timestamp = str_to_datetime(lastmod, context='lastmod')

+    @property
+    def datetime(self):
+        """The datetime of the resource change in W3C Datetime syntax, Z notation."""
+        return datetime_to_str(self.ts_datetime)
+
+    @datetime.setter
+    def datetime(self, datetime):
+        """Set timestamp from a W3C Datetime Last-Modified value."""
+        self.ts_datetime = str_to_datetime(datetime, context='ts_datetime')
+
+    @property
+    def ts_at(self):
+        """ts_at is the timestamp for md_at."""
+        return self._get_extra('ts_at')
+
    @property
    def md_at(self):
        """md_at values in W3C Datetime syntax, Z notation."""
@ -194,6 +216,11 @@ class Resource(object):
            'ts_at',
            str_to_datetime(md_at, context='md_at datetime'))

+    @property
+    def ts_completed(self):
+        """ts_completed is the timestamp for md_completed."""
+        return self._get_extra('ts_completed')
+
    @property
    def md_completed(self):
        """md_completed value in W3C Datetime syntax, Z notation."""
@ -206,6 +233,11 @@ class Resource(object):
            'ts_completed',
            str_to_datetime(md_completed, context='md_completed datetime'))

+    @property
+    def ts_from(self):
+        """ts_from is the timestamp for md_from."""
+        return self._get_extra('ts_from')
+
    @property
    def md_from(self):
        """md_from value in W3C Datetime syntax, Z notation."""
@ -218,6 +250,11 @@ class Resource(object):
            'ts_from',
            str_to_datetime(md_from, context='md_from datetime'))

+    @property
+    def ts_until(self):
+        """ts_until is the timestamp for md_until."""
+        return self._get_extra('ts_until')
+
    @property
    def md_until(self):
        """md_until value in W3C Datetime syntax, Z notation."""
@ -260,8 +297,10 @@ class Resource(object):
        return(None)

    @hash.setter
-    def hash(self, hash):
-        """Parse space separated set of values.
+    def hash(self, hashes_str):
+        """Parse space separated set of values to set md5, sha1 and sha256.
+
+        Any existing values for md5, sha1 and sha256 will be removed first.

        See specification at:
        http://tools.ietf.org/html/draft-snell-atompub-link-extensions-09
@ -270,27 +309,24 @@ class Resource(object):
        self.md5 = None
        self.sha1 = None
        self.sha256 = None
-        if (hash is None):
-            return
        hash_seen = set()
        errors = []
-        for entry in hash.split():
+        for entry in hashes_str.split():
            (hash_type, value) = entry.split(':', 1)
-            if (hash_type in hash_seen):
-                errors.append("Ignored duplicate hash type %s" % (hash_type))
+            if hash_type in hash_seen:
+                errors.append("duplicate hash type %s" % (hash_type))
            else:
                hash_seen.add(hash_type)
-                if (hash_type == 'md5'):
+                if hash_type == 'md5':
                    self.md5 = value
-                elif (hash_type == 'sha-1'):
+                elif hash_type == 'sha-1':
                    self.sha1 = value
-                elif (hash_type == 'sha-256'):
+                elif hash_type == 'sha-256':
                    self.sha256 = value
                else:
-                    errors.append("Ignored unsupported hash type (%s)" %
-                                  (hash_type))
-        if (len(errors) > 0):
-            raise ValueError(". ".join(errors))
+                    errors.append("unsupported hash type %s" % (hash_type))
+        if len(errors) > 0:
+            raise ValueError("Ignored " + ", ".join(errors))

    def link(self, rel):
        """Look for link with specified rel, return else None.
@ -299,19 +335,16 @@ class Resource(object):
        specified rel value. If there are multiple links with the
        same rel then just the first will be returned
        """
-        if (self.ln is None):
-            return(None)
-        for link in self.ln:
-            if ('rel' in link and link['rel'] == rel):
-                return(link)
-        return(None)
+        if self.ln is not None:
+            for link in self.ln:
+                if 'rel' in link and link['rel'] == rel:
+                    return link
+        return None

    def link_href(self, rel):
        """Look for link with specified rel, return href from it or None."""
        link = self.link(rel)
-        if (link is not None):
-            link = link['href']
-        return(link)
+        return None if link is None else link['href']

    def link_set(self, rel, href, allow_duplicates=False, **atts):
        """Set/create link with specified rel, set href and any other attributes.
@ -325,13 +358,13 @@ class Resource(object):
        Be aware that adding links to a Resource object will
        significantly increase the size of the object.
        """
-        if (self.ln is None):
+        if self.ln is None:
            # automagically create a self.ln list
            self.ln = []
            link = None
        else:
            link = self.link(rel)
-        if (link is not None and not allow_duplicates):
+        if link is not None and not allow_duplicates:
            # overwrite current value
            link['href'] = href
        else:
@ -381,7 +414,7 @@ class Resource(object):
    @property
    def contents(self):
        """Get the URI of and ResourceSync rel="contents" link."""
-        return(self.link_href('index'))
+        return(self.link_href('contents'))

    @contents.setter
    def contents(self, uri, type='application/xml'):
@ -403,36 +436,43 @@ class Resource(object):
    def __eq__(self, other):
        """Equality test for resources allowing <1s difference in timestamp.

-        See equal(...) for more details of equality test
+        See equal(...) for more details of equality test.
        """
        return(self.equal(other, delta=1.0))

    def equal(self, other, delta=0.0):
        """Equality or near equality test for resources.

-        Equality means:
+        Equality means that any parameters that exist for both resources match,
+        essentially "no proof of inequality":
        1. same uri, AND
        2. same timestamp WITHIN delta if specified for either, AND
-        3. same md5 if specified for both, AND
+        3. same md5|sha1|sha256 if specified for both, AND
        4. same length if specified for both
        """
-        if (other is None):
+        if other is None:
+            return False
+        if self.uri != other.uri:
            return False
-        if (self.uri != other.uri):
-            return(False)
        if (self.timestamp is not None or other.timestamp is not None):
            # not equal if only one timestamp specified
-            if (self.timestamp is None or
-                    other.timestamp is None or
-                    abs(self.timestamp - other.timestamp) >= delta):
-                return(False)
-        if ((self.md5 is not None and other.md5 is not None) and
-                self.md5 != other.md5):
-            return(False)
-        if ((self.length is not None and other.length is not None) and
-                self.length != other.length):
-            return(False)
-        return(True)
+            if (self.timestamp is None
+                    or other.timestamp is None
+                    or abs(self.timestamp - other.timestamp) >= delta):
+                return False
+        if ((self.md5 is not None and other.md5 is not None)
+                and self.md5 != other.md5):
+            return False
+        if ((self.sha1 is not None and other.sha1 is not None)
+                and self.sha1 != other.sha1):
+            return False
+        if ((self.sha256 is not None and other.sha256 is not None)
+                and self.sha256 != other.sha256):
+            return False
+        if ((self.length is not None and other.length is not None)
+                and self.length != other.length):
+            return False
+        return True

    def __str__(self):
        """Return a human readable string for this resource.
@ -441,9 +481,12 @@ class Resource(object):
        designed to support logging.
        """
        s = [str(self.uri), str(self.lastmod), str(self.length),
-             str(self.md5 if self.md5 else self.sha1)]
+             str(self.md5 if self.md5 else (self.sha1 if self.sha1 else self.sha256))]
        if (self.change is not None):
-            s.append(str(self.change))
+            ch = str(self.change)
+            if self.datetime is not None:
+                ch += " @ " + str(self.datetime)
+            s.append(ch)
        if (self.path is not None):
            s.append(str(self.path))
        return "[ " + " | ".join(s) + " ]"
--- a/resync/resource_container.py
+++ b/resync/resource_container.py
@ -10,10 +10,26 @@ base class for ResourceList, ChangeList, etc. This class provides
 only the data storage and manipulation, the ListBase class
 adds IO.
 """
-import collections
+import collections.abc
 from .w3c_datetime import datetime_to_str


+def _str_datetime_now(x='now'):
+    """Return datetime string for use with time attributes.
+
+    Handling depends on input:
+      'now'   - returns datetime for now
+      other string - no change, return same value
+      otherwise - assume datetime value, generate string
+    """
+    if (x == 'now' or not isinstance(x, str)):
+        # Now, this is wht datetime_to_str() with no arg gives
+        return datetime_to_str(x)
+    else:
+        # Assume valid datetime string
+        return x
+
+
 class ResourceContainer(object):
    """Class containing resource-like objects.

@ -26,6 +42,11 @@ class ResourceContainer(object):
    - uri is optional identifier of this container object
    - capability_name - name of this capability

+    The properties md_from, md_at, md_until and md_completed are datetime
+    strings. The setters acceopt either a string value (not checked), the value
+    'now' to generate the current datetime, or otherwise an integer timestamp
+    value that is converted to a string.
+
    Derived classes may add extra functionality such as len() etc..
    However, any code designed to work with any ResourceContainer
    should use only the core functionality.
@ -45,19 +66,16 @@ class ResourceContainer(object):

        Baseline implementation use iterator given by resources property.
        """
-        return(iter(self.resources))
+        return iter(self.resources)

    def __getitem__(self, index):
        """Feed through for __getitem__ of resources property."""
-        return(self.resources[index])
+        return self.resources[index]

    @property
    def capability(self):
-        """Get/set the <rs:md capability="" .../> attribute."""
-        if ('capability' in self.md):
-            return(self.md['capability'])
-        else:
-            return(None)
+        """Get/set the <rs:md capability="" .../> attribute, None if not set."""
+        return self.md.get('capability')

    @capability.setter
    def capability(self, capability):
@ -65,66 +83,53 @@ class ResourceContainer(object):

    @property
    def md_from(self):
-        """Get/set the <rs:md from="" .../> attribute."""
-        if ('md_from' in self.md):
-            return(self.md['md_from'])
-        else:
-            return(None)
+        """Get/set the <rs:md from="" .../> attribute, None if not set."""
+        return self.md.get('md_from')

    @md_from.setter
    def md_from(self, md_from):
-        self.md['md_from'] = self._str_datetime_now(md_from)
+        self.md['md_from'] = _str_datetime_now(md_from)

    @property
    def md_until(self):
-        """Get/set the <rs:md until="" .../> attribute."""
-        if ('md_until' in self.md):
-            return(self.md['md_until'])
-        else:
-            return(None)
+        """Get/set the <rs:md until="" .../> attribute, None if not set."""
+        return self.md.get('md_until')

    @md_until.setter
    def md_until(self, md_until):
-        self.md['md_until'] = self._str_datetime_now(md_until)
+        self.md['md_until'] = _str_datetime_now(md_until)

    @property
    def md_at(self):
        """Get/set the <rs:md at="" attribute."""
-        if ('md_at' in self.md):
-            return(self.md['md_at'])
-        else:
-            return(None)
+        return self.md.get('md_at')

    @md_at.setter
    def md_at(self, md_at):
-        self.md['md_at'] = self._str_datetime_now(md_at)
+        self.md['md_at'] = _str_datetime_now(md_at)

    @property
    def md_completed(self):
-        """Get/set the <rs:md completed="" .../> attribute."""
-        if ('md_completed' in self.md):
-            return(self.md['md_completed'])
-        else:
-            return(None)
+        """Get/set the <rs:md completed="" .../> attribute, None if not set."""
+        return self.md.get('md_completed')

    @md_completed.setter
    def md_completed(self, md_completed):
-        self.md['md_completed'] = self._str_datetime_now(md_completed)
+        self.md['md_completed'] = _str_datetime_now(md_completed)

    def link(self, rel):
        """Look for link with specified rel, return else None."""
        for link in self.ln:
-            if ('rel' in link and
-                    link['rel'] == rel):
-                return(link)
-        return(None)
+            if ('rel' in link and link['rel'] == rel):
+                return link
+        return None

    def link_href(self, rel):
        """Look for link with specified rel, return href from it or None."""
        link = self.link(rel)
        if (link is not None):
            link = link['href']
-        return(link)
+        return link

    def link_set(self, rel, href, **atts):
        """Set/create link with specified rel, set href and any other attributes.
@ -133,16 +138,16 @@ class ResourceContainer(object):
        also defines the type attributes and others are permitted also. See
        description of allowed formats in

-        http://www.openarchives.org/rs/resourcesync.html#DocumentFormats
+        http://www.openarchives.org/rs/resourcesync#DocumentFormats
        """
        link = self.link(rel)
-        if (link is not None):
-            # overwrite current value
-            link['href'] = href
-        else:
+        if link is None:
            # create new link
            link = {'rel': rel, 'href': href}
            self.ln.append(link)
+        else:
+            # overwrite current value
+            link['href'] = href
        for k in atts:
            link[k] = atts[k]

@ -190,7 +195,7 @@ class ResourceContainer(object):

        Must be implemented in derived class.
        """
-        if isinstance(resource, collections.Iterable):
+        if isinstance(resource, collections.abc.Iterable):
            for r in resource:
                self.resources.append(r)
        else:
@ -203,24 +208,6 @@ class ResourceContainer(object):
            uris.append(r.uri)
        return(uris)

-    def prune_before(self, timestamp):
-        """Remove all resources with timestamp earlier than that given.
-
-        Returns the number of entries removed. Will raise an excpetion
-        if there are any entries without a timestamp.
-        """
-        n = 0
-        pruned = []
-        for r in self.resources:
-            if (r.timestamp is None):
-                raise Exception("Entry %s has no timestamp" % (r.uri))
-            elif (r.timestamp >= timestamp):
-                pruned.append(r)
-            else:
-                n += 1
-        self.resources = pruned
-        return(n)
-
    def prune_dupes(self):
        """Remove all but the last entry for a given resource URI.

@ -251,29 +238,3 @@ class ResourceContainer(object):
                pruned2.append(r)
        self.resources = pruned2
        return(n)
-
-    def __str__(self):
-        """Return string of all resources in order given by interator."""
-        s = ''
-        for resource in self:
-            s += str(resource) + "\n"
-        return(s)
-
-    def _str_datetime_now(self, x=None):
-        """Return datetime string for use with time attributes.
-
-        Handling depends on input:
-          'now'   - returns datetime for now
-          number  - assume datetime values, generate string
-          other   - no change, return same value
-        """
-        if (x == 'now'):
-            # Now, this is wht datetime_to_str() with no arg gives
-            return(datetime_to_str())
-        try:
-            # Test for number
-            junk = x + 0.0
-            return datetime_to_str(x)
-        except TypeError:
-            # Didn't look like a number, treat as string
-            return x
--- a/resync/resource_dump.py
+++ b/resync/resource_dump.py
@ -24,10 +24,12 @@ class ResourceDump(ResourceList):
    """

    def __init__(self, resources=None, md=None, ln=None, uri=None,
-                 allow_multifile=None, mapper=None):
+                 allow_multifile=None, mapper=None,
+                 spec_version='1.1', add_lastmod=False):
        """Initialize ResourceDump."""
        super(ResourceDump, self).__init__(
-            resources=resources, md=md, ln=ln, uri=uri, mapper=mapper)
+            resources=resources, md=md, ln=ln, uri=uri, mapper=mapper,
+            spec_version=spec_version, add_lastmod=add_lastmod)
        self.capability_name = 'resourcedump'

    def write(self, basename="/tmp/resource_dump.xml"):
--- a/resync/resource_dump_manifest.py
+++ b/resync/resource_dump_manifest.py
@ -21,14 +21,10 @@ class ResourceDumpManifest(ResourceList):
    """

    def __init__(self, resources=None, md=None, ln=None,
-                 uri=None, allow_multifile=None, mapper=None):
+                 uri=None, allow_multifile=None, mapper=None,
+                 spec_version='1.1', add_lastmod=False):
        """Initialize ResourceDumpManifest."""
-        super(
-            ResourceDumpManifest,
-            self).__init__(
-            resources=resources,
-            md=md,
-            ln=ln,
-            uri=uri,
-            mapper=mapper)
+        super(ResourceDumpManifest, self).__init__(
+            resources=resources, md=md, ln=ln, uri=uri, mapper=mapper,
+            spec_version=spec_version, add_lastmod=add_lastmod)
        self.capability_name = 'resourcedump-manifest'
--- a/resync/resource_list.py
+++ b/resync/resource_list.py
@ -1,31 +1,28 @@
 """ResourceSync Resource List object.

 A Resource List is a set of resources with some metadata for
-each resource. Comparison of resource lists from a source and
-a destination allows understanding of whether the two are in
-sync or whether some resources need to be updated at the
-destination.
+each resource. Every resource must have a uri attribute.
+Comparison of resource lists from a source and a destination
+allows understanding of whether the two are in sync or whether
+some resources need to be updated at the destination.

 There may also be metadata about the Resource List, and links
 to other ResourceSync documents. Metadata include the
 timestamp of the Resource List (md_at) and, optionally, the
 timestamp when creation of the Resource List was completed
-(md_completed).at the top level. These include a creation timestamp
+(md_completed) at the top level. These include a creation timestamp
 (from) and links to the Capability List.

 Described in specification at:
 http://www.openarchives.org/rs/resourcesync#DescResources
 """

-import collections
+import collections.abc
+from collections import OrderedDict
 import os
 from datetime import datetime
 import re
 import sys
-try:  # python3
-    from urllib.request import URLopener
-except ImportError:  # python2
-    from urllib import URLopener

 from .list_base_with_index import ListBaseWithIndex
 from .sitemap import Sitemap
@ -33,29 +30,62 @@ from .mapper import Mapper, MapperError
 from .url_authority import UrlAuthority


-class ResourceListDict(dict):
+class SortedIterMixin(object):
+    """Mixin to provide sorted_iter() method."""
+
+    def sorted_iter(self):
+        """Iterator over all the resources in this dict by sorted key order."""
+        self._sorted_iter_next_list = sorted(self.keys(), reverse=True)
+        return iter(self._sorted_iter_next, None)
+
+    def _sorted_iter_next(self):
+        if (len(self._sorted_iter_next_list) > 0):
+            return self[self._sorted_iter_next_list.pop()]
+        else:
+            return None
+
+
+class ResourceListDict(dict, SortedIterMixin):
    """Default implementation of class to store resources in ResourceList.

    Key properties of this class are:
    - has add(resource) method
-    - is iterable and results given in alphanumeric order by resource.uri
+    - is iterable and gives resources (not keys) in alphanumeric order by
+      resource.uri
    """

    def __iter__(self):
        """Iterator over all the resources in this ResourceListDict."""
-        self._iter_next_list = sorted(self.keys())
-        self._iter_next_list.reverse()
-        return(iter(self._iter_next, None))
-
-    def _iter_next(self):
-        if (len(self._iter_next_list) > 0):
-            return(self[self._iter_next_list.pop()])
-        else:
-            return(None)
+        return self.sorted_iter()

    def uris(self):
        """Extract sorted list of URIs for resources in this ResourceListDict."""
-        return(sorted(self.keys()))
+        return sorted(self.keys())
+
+    def add(self, resource, replace=False):
+        """Add just a single resource."""
+        uri = resource.uri
+        if (uri in self.keys() and not replace):
+            raise ResourceListDupeError(
+                "Attempt to add resource already in resource_list")
+        self[uri] = resource
+
+
+class ResourceListOrdered(OrderedDict, SortedIterMixin):
+    """Alternative implementation of class to store resources in ResourceList.
+
+    Key properties of this class are:
+    - has add(resource) method
+    - is iterable and gives resources (not keys) in the order added
+    """
+
+    def __iter__(self):
+        """Iterator over the resources in the list in the order added."""
+        return iter(self.values())
+
+    def uris(self):
+        """Extract list of all resource URIs in the order added."""
+        return self.keys()

    def add(self, resource, replace=False):
        """Add just a single resource."""
@ -66,42 +96,6 @@ class ResourceListDict(dict):
        self[uri] = resource


-class ResourceListOrdered(list):
-    """Alternative implementation of class to store resources in ResourceList.
-
-    FIXME - This is a rather inefficient implementation which involves
-    scanning all resources to check for duplicates. Designed just to enable
-    re-creation of examples in the spec. Something dictionary based would
-    likely be better. Might be best to use OrderedDict but that is available
-    natively only in python >= 2.7 and this library is designed for 2.6,2.7.
-
-    Key properties of this class are:
-    - has add(resource) method
-    - is iterable and results given in order added (not the usual one!)
-    """
-
-    def uris(self):
-        """Extract list of all resource URIs (in the order added)."""
-        uris = []
-        for r in self:
-            uris.append(r.uri)
-        return(uris)
-
-    def add(self, resource, replace=False):
-        """Add a single resource, check for dupes."""
-        uri = resource.uri
-        for r in self:
-            if (uri == r.uri):
-                if (replace):
-                    r = resource
-                    return
-                else:
-                    raise ResourceListDupeError(
-                        "Attempt to add resource already in resource_list")
-        # didn't find it in list, add to end
-        self.append(resource)
-
-
 class ResourceListDupeError(Exception):
    """Exception in case of duplicate resource."""

@ -127,11 +121,13 @@ class ResourceList(ListBaseWithIndex):

    The default storage is unordered but the iterator imposes a canonical
    order which is alphabetical by URI. If it is desired to have
-    resources listed in the order they are added then the ResourceDictOrdered
+    resources listed in the order they are added then the ResourceListOrdered
    class may be specified on creation:

        rl = ResourceList( resources_class=ResourceDictOrdered )

+    Use of ResourceListOrdered will be faster than the default.
+
    In normal use it is expected that any Resource List Index will be
    created automatically when writing out a large Resource List in
    multiple sitemap files. However, should it be necessary to
@ -144,12 +140,21 @@ class ResourceList(ListBaseWithIndex):
    """

    def __init__(self, resources=None, count=None, md=None, ln=None, uri=None,
-                 allow_multifile=None, mapper=None, resources_class=ResourceListDict):
+                 allow_multifile=None, mapper=None,
+                 spec_version='1.1', add_lastmod=False,
+                 resources_class=ResourceListDict):
        """Initialize ResourceList."""
-        super(ResourceList, self).__init__(resources=resources, count=count, md=md, ln=ln, uri=uri,
-                                           capability_name='resourcelist',
-                                           allow_multifile=allow_multifile, mapper=mapper,
-                                           resources_class=resources_class)
+        super(ResourceList, self).__init__(
+            resources=resources,
+            count=count,
+            md=md,
+            ln=ln,
+            uri=uri,
+            capability_name='resourcelist',
+            allow_multifile=allow_multifile,
+            mapper=mapper, spec_version=spec_version,
+            add_lastmod=add_lastmod,
+            resources_class=resources_class)

    def add(self, resource, replace=False):
        """Add a resource or an iterable collection of resources.
@ -157,7 +162,7 @@ class ResourceList(ListBaseWithIndex):
        Will throw a ValueError if the resource (ie. same uri) already
        exists in the ResourceList, unless replace=True.
        """
-        if isinstance(resource, collections.Iterable):
+        if isinstance(resource, collections.abc.Iterable):
            for r in resource:
                self.resources.add(r, replace)
        else:
@ -171,11 +176,11 @@ class ResourceList(ListBaseWithIndex):
        written to work for any objects in self and sc, provided that the
        == operator can be used to compare them.

-        The functioning of this method depends on the iterators for self and
-        src providing access to the resource objects in URI order.
+        The functioning of this method depends on the sorted_iter() iterators
+        for self and src providing access to the resource objects in URI order.
        """
-        dst_iter = iter(self.resources)
-        src_iter = iter(src.resources)
+        dst_iter = self.resources.sorted_iter()
+        src_iter = src.resources.sorted_iter()
        same = ResourceList()
        updated = ResourceList()
        deleted = ResourceList()
@ -184,21 +189,19 @@ class ResourceList(ListBaseWithIndex):
        src_cur = next(src_iter, None)
        while ((dst_cur is not None) and (src_cur is not None)):
            # print 'dst='+dst_cur+'  src='+src_cur
-            if (dst_cur.uri == src_cur.uri):
+            if dst_cur.uri == src_cur.uri:
                if (dst_cur == src_cur):
                    same.add(dst_cur)
                else:
                    updated.add(src_cur)
                dst_cur = next(dst_iter, None)
                src_cur = next(src_iter, None)
-            elif (not src_cur or dst_cur.uri < src_cur.uri):
+            elif dst_cur.uri < src_cur.uri:
                deleted.add(dst_cur)
                dst_cur = next(dst_iter, None)
-            elif (not dst_cur or dst_cur.uri > src_cur.uri):
+            else:  # dst_cur.uri > src_cur.uri:
                created.add(src_cur)
                src_cur = next(src_iter, None)
-            else:
-                raise Exception("this should not be possible")
        # what do we have leftover in src or dst lists?
        while (dst_cur is not None):
            deleted.add(dst_cur)
--- a/resync/resource_list_builder.py
+++ b/resync/resource_list_builder.py
@ -3,13 +3,8 @@
 import os
 import os.path
 import re
-import sys
 import time
 import logging
-try:  # python3
-    from urllib.request import URLopener
-except ImportError:  # python2
-    from urllib import URLopener
 from defusedxml.ElementTree import parse

 from .hashes import Hashes
@ -28,11 +23,10 @@ class ResourceListBuilder():
    - set_hashes to indicate which hashes should be calculated for each resource
    - set_path set true to add path attribute for each resource
    - set_length set true to include file length in resource_list (defaults true)
-    - exclude_dirs is a list of directory names to exclude
-      (defaults to ['CVS','.git'))
-
-    FIXME - should add options to set sha1 and sha256 in addition or as
-    alternatives to md5.
+    - exclude_patterns is a list of files and directory names patterns to exclude
+      using re.match(..). These patterns are left anchored so thus need to be
+      preceded with .* if there may be arbitrary leading characters (defaults to
+      empty)
    """

    def __init__(self, mapper=None, set_hashes=None,
@ -44,9 +38,10 @@ class ResourceListBuilder():

        The following attributes may be set to determine information added to
        each Resource object based on the disk scan:
-        - set_hashes - Set of digests to computer for each resource. This may add
+        - set_hashes - Set of digests to compute for each resource. This may add
        significant time to the scan process as each file has to be read to
-        compute the hash. Empty set or None means no hashes calculated
+        compute the hash. Empty set or None means no hashes calculated, names
+        defined in Hashes object
        - set_length - False to not add length for each resources
        - set_path - True to add local path information for each file/resource
        """
@ -54,34 +49,38 @@ class ResourceListBuilder():
        self.set_path = set_path
        self.set_hashes = set_hashes if (set_hashes and len(set_hashes) > 0) else None
        self.set_length = set_length
-        self.exclude_files = ['sitemap\d{0,5}.xml']
-        self.exclude_dirs = ['CVS', '.git']
+        self.exclude_patterns = []
        self.include_symlinks = False
+        self.log_count_increment = 50000  # Write log message after 50000 files
        # Used internally only:
        self.logger = logging.getLogger('resync.resource_list_builder')
-        self.compiled_exclude_files = []
+        self.compiled_exclude_patterns = []

-    def add_exclude_files(self, exclude_patterns):
-        """Add more patterns of files to exclude while building resource_list."""
+    def add_exclude_patterns(self, exclude_patterns):
+        """Add more patterns of files or directories to exclude while building resource_list.
+
+        After patterns are added the set of compiled patterns is updated.
+        """
        for pattern in exclude_patterns:
-            self.exclude_files.append(pattern)
+            self.exclude_patterns.append(pattern)
+        self._compile_excludes()

-    def compile_excludes(self):
-        """Compile a set of regexps for files to be exlcuded from scans."""
-        self.compiled_exclude_files = []
-        for pattern in self.exclude_files:
+    def _compile_excludes(self):
+        # Compile a set of regexps for files and directories to be exlcuded from scans
+        self.compiled_exclude_patterns = []
+        for pattern in self.exclude_patterns:
            try:
-                self.compiled_exclude_files.append(re.compile(pattern))
+                self.compiled_exclude_patterns.append(re.compile(pattern))
            except re.error as e:
                raise ValueError(
-                    "Bad python regex in exclude '%s': %s" % (pattern, str(e)))
+                    "Bad python regex in exclude pattern '%s': %s" % (pattern, str(e)))

-    def exclude_file(self, file):
-        """True if file should be exclude based on name pattern."""
-        for pattern in self.compiled_exclude_files:
-            if (pattern.match(file)):
-                return(True)
-        return(False)
+    def _exclude(self, file):
+        # True if file should be exclude based on name pattern.
+        for pattern in self.compiled_exclude_patterns:
+            if pattern.match(file):
+                return True
+        return False

    def from_disk(self, resource_list=None, paths=None):
        """Create or extend resource_list with resources from disk scan.
@ -112,8 +111,6 @@ class ResourceListBuilder():
        # Either use resource_list passed in or make a new one
        if (resource_list is None):
            resource_list = ResourceList()
-        # Compile exclude pattern matches
-        self.compile_excludes()
        # Work out start paths from map if not explicitly specified
        if (paths is None):
            paths = []
@ -138,23 +135,24 @@ class ResourceListBuilder():
            raise ValueError("Must specify path, resource_list and mapper")
        # is path a directory or a file? for each file: create Resource object,
        # add, increment counter
-        if (sys.version_info < (3, 0)):
-            path = path.decode('utf-8')
        if os.path.isdir(path):
            num_files = 0
            for dirpath, dirs, files in os.walk(path, topdown=True):
                for file_in_dirpath in files:
                    num_files += 1
-                    if (num_files % 50000 == 0):
+                    if (num_files % self.log_count_increment == 0):
                        self.logger.info(
                            "ResourceListBuilder.from_disk_add_path: %d files..." % (num_files))
                    self.add_file(resource_list=resource_list,
                                  dir=dirpath, file=file_in_dirpath)
-                    # prune list of dirs based on self.exclude_dirs
-                    for exclude in self.exclude_dirs:
-                        if exclude in dirs:
-                            self.logger.debug("Excluding dir %s" % (exclude))
-                            dirs.remove(exclude)
+                # prune list of dirs based on self.exclude_dirs
+                prune = []
+                for dir in dirs:
+                    if self._exclude(dir):
+                        self.logger.debug("Excluding dir '%s'" % (dir))
+                        prune.append(dir)
+                for dir in prune:
+                    dirs.remove(dir)
        else:
            # single file
            self.add_file(resource_list=resource_list, file=path)
@ -164,37 +162,27 @@ class ResourceListBuilder():

        Follows object settings of set_path, set_hashes and set_length.
        """
+        if self._exclude(file):
+            self.logger.debug("Excluding file '%s'" % (file))
+            return
+        # get abs filename and also URL
+        if (dir is not None):
+            file = os.path.join(dir, file)
+        if os.path.islink(file) and not self.include_symlinks:
+            self.logger.warning("Ignoring symlink '%s'" % (file))
+            return
        try:
-            if self.exclude_file(file):
-                self.logger.debug("Excluding file %s" % (file))
-                return
-            # get abs filename and also URL
-            if (dir is not None):
-                file = os.path.join(dir, file)
-            if (not os.path.isfile(file) or not (
-                    self.include_symlinks or not os.path.islink(file))):
-                return
-            uri = self.mapper.dst_to_src(file)
-            if (uri is None):
-                raise Exception("Internal error, mapping failed")
+            uri = self.mapper.dst_to_src(file)  # might throw MapperError
            file_stat = os.stat(file)
        except OSError as e:
-            sys.stderr.write("Ignoring file %s (error: %s)" % (file, str(e)))
+            self.logger.warning("Ignoring file '%s' (error: %s)" % (file, str(e)))
            return
        timestamp = file_stat.st_mtime  # UTC
        r = Resource(uri=uri, timestamp=timestamp)
-        if (self.set_path):
-            # add full local path
+        if self.set_path:  # add full local path
            r.path = file
-        if (self.set_hashes):
-            hasher = Hashes(self.set_hashes, file)
-            if ('md5' in self.set_hashes):
-                r.md5 = hasher.md5
-            if ('sha-1' in self.set_hashes):
-                r.sha1 = hasher.sha1
-            if ('sha-256' in self.set_hashes):
-                r.sha256 = hasher.sha256
-        if (self.set_length):
-            # add length
+        if self.set_hashes:  # add any hashes requested
+            Hashes(self.set_hashes, file).set(r)
+        if self.set_length:  # add length
            r.length = file_stat.st_size
        resource_list.add(r)
--- a/resync/sitemap.py
+++ b/resync/sitemap.py
@ -1,17 +1,11 @@
 """Read and write ResourceSync documents as sitemaps."""

-import re
+import io
+import logging
 import os
 import sys
-import logging
 from defusedxml.ElementTree import parse
 from xml.etree.ElementTree import ElementTree, Element, tostring
-try:  # python2
-    # Must try this first as io also exists in python2
-    # but in the wrong one!
-    import StringIO as io
-except ImportError:  # python3
-    import io

 from .resource import Resource
 from .resource_container import ResourceContainer
@ -40,7 +34,7 @@ class SitemapIndexError(Exception):
        self.message = message
        self.etree = etree

-    def __repr__(self):
+    def __str__(self):
        """Return just the message attribute."""
        return(self.message)

@ -68,17 +62,45 @@ class Sitemap(object):
    of a sitemapindex and multiple sitemap documents. Instead it will
    take a default and throw and exception if the other case is found
    so that the calling code can handle it.
+
+    Default is to follow ResourceSync v1.1 which distinguishes the meanings
+    of the <lastmod> element (resource modified time as used for Last-Modified
+    header) and the newly introduced <rs:md datetime="..."> attribute (the
+    change time in ChangeList and similar resources). There are two parameters
+    to control these behaviors:
+
+        spec_version - set to either '1.0' or '1.1'. If set to '1.0' then no
+            datetime attribute will be written or parsed
+        add_lastmod - set True to add a lastmod with the same value of datetime
+            if no specific lastmod is specified. Applies only when spec_version
+            '1.1' is selected, provide compatibility with systems that do not
+            understand datetime but instead rely on lastmod
    """

-    def __init__(self, pretty_xml=False):
+    def __init__(self, pretty_xml=False, spec_version='1.1', add_lastmod=False):
        """Initialize Sitemap object."""
        self.logger = logging.getLogger('resync.sitemap')
        self.pretty_xml = pretty_xml
+        self.spec_1_0 = (spec_version == '1.0')  # v1.0 else assume v1.1
+        self.add_lastmod = add_lastmod  # Optional in v1.1
        # Classes used when parsing
        self.resource_class = Resource
        # Information recorded for logging
        self.resources_created = 0    # Set during parsing sitemap
        self.parsed_index = None      # Set True for sitemapindex, False for sitemap
+        # rs:md attributes
+        self.md_att_keys = ['md_at', 'capability', 'change', 'datetime',
+                            'md_completed', 'md_from', 'hash', 'length',
+                            'path', 'mime_type', 'md_until']
+        # capabilities
+        self.capabilities = ['resourcelist', 'changelist', 'resourcedump',
+                             'changedump', 'resourcedump-manifest',
+                             'changedump-manifest', 'capabilitylist',
+                             'description',
+                             'resourcelist-archive', 'resourcedump-archive',
+                             'changelist-archive', 'changedump-archive']
+        if self.spec_1_0:
+            self.md_att_keys.remove('datetime')

    # Write the XML for a sitemap or sitemapindex

@ -117,25 +139,13 @@ class Sitemap(object):
        if (fh is None):
            xml_buf = io.StringIO()
            fh = xml_buf
-        if (sys.version_info >= (3, 0)):
-            tree.write(
-                fh,
-                encoding='unicode',
-                xml_declaration=True,
-                method='xml')
-        elif (sys.version_info >= (2, 7)):
-            tree.write(
-                fh,
-                encoding='UTF-8',
-                xml_declaration=True,
-                method='xml')
-        else:  # python2.6
-            tree.write(fh, encoding='UTF-8')
+        tree.write(
+            fh,
+            encoding='unicode',
+            xml_declaration=True,
+            method='xml')
        if (xml_buf is not None):
-            if (sys.version_info >= (3, 0)):
-                return(xml_buf.getvalue())
-            else:
-                return(xml_buf.getvalue().decode('utf-8'))
+            return(xml_buf.getvalue())

    # Read/parse an XML sitemap or sitemapindex

@ -193,18 +203,14 @@ class Sitemap(object):
        in_preamble = True
        self.resources_created = 0
        seen_top_level_md = False
-        for e in etree.getroot().getchildren():
-            # look for <rs:md> and <rs:ln>, first <url> ends
-            # then look for resources in <url> blocks
+        for e in list(etree.getroot()):
+            # look for <rs:md> and <rs:ln>, first <url>/<sitemap> ends
+            # then look for resources in <url>/<sitemap> blocks.
+            # ignore any elements we don't recognize
            if (e.tag == resource_tag):
                in_preamble = False  # any later rs:md or rs:ln is error
                r = self.resource_from_etree(e, self.resource_class)
-                try:
-                    resources.add(r)
-                except SitemapDupeError:
-                    self.logger.warning(
-                        "dupe of: %s (lastmod=%s)" %
-                        (r.uri, r.lastmod))
+                resources.add(r)
                self.resources_created += 1
            elif (e.tag == "{" + RS_NS + "}md"):
                if (in_preamble):
@ -223,9 +229,6 @@ class Sitemap(object):
                else:
                    raise SitemapParseError(
                        "Found <rs:ln> after first <url> in sitemap")
-            else:
-                # element we don't recognize, ignore
-                pass
        # check that we read to right capability document
        if (capability is not None):
            if ('capability' not in resources.md):
@ -255,23 +258,25 @@ class Sitemap(object):
        sub = Element('loc')
        sub.text = resource.uri
        e.append(sub)
-        if (resource.timestamp is not None):
-            # Create appriate element for timestamp
-            sub = Element('lastmod')
-            sub.text = str(resource.lastmod)  # W3C Datetime in UTC
-            e.append(sub)
+        lm = resource.lastmod  # W3C Datetime in UTC
+        if lm is not None or self.spec_1_0 or self.add_lastmod:
+            # In 1.0 we either use the lastmod specified or else use the
+            # datetime value because there should always be a lastmod
+            if lm is None and (self.spec_1_0 or self.add_lastmod):
+                lm = resource.datetime  # W3C Datetime in UTC
+            if lm is not None:
+                sub = Element('lastmod')
+                sub.text = lm
+                e.append(sub)
        md_atts = {}
-        for att in ('capability', 'change', 'hash', 'length', 'path', 'mime_type',
-                    'md_at', 'md_completed', 'md_from', 'md_until'):
+        for att in self.md_att_keys:
            val = getattr(resource, att, None)
            if (val is not None):
-                md_atts[self._xml_att_name(att)] = str(val)
+                md_atts[att] = str(val)
        if (len(md_atts) > 0):
-            md = Element('rs:md', md_atts)
-            e.append(md)
+            self.add_element_with_atts_to_etree(e, 'rs:md', md_atts, add_return=False)
        # add any <rs:ln>
-        if (hasattr(resource, 'ln') and
-                resource.ln is not None):
+        if (hasattr(resource, 'ln') and resource.ln is not None):
            for ln in resource.ln:
                self.add_element_with_atts_to_etree(e, 'rs:ln', ln)
        if (self.pretty_xml):
@ -282,19 +287,11 @@ class Sitemap(object):
        """Return string for the resource as part of an XML sitemap.

        Returns a string with the XML snippet representing the resource,
-        without any XML declaration.
+        without any XML declaration. (So much simpler now only Python 3.x
+        supported, see earlier versions for 2.6, 2.7 etc.)
        """
        e = self.resource_etree_element(resource)
-        if (sys.version_info >= (3, 0)):
-            # python3.x
-            return(tostring(e, encoding='unicode', method='xml'))
-        elif (sys.version_info >= (2, 7)):
-            s = tostring(e, encoding='UTF-8', method='xml')
-        else:
-            # must not specify method='xml' in python2.6
-            s = tostring(e, encoding='UTF-8')
-        # Chop off XML declaration that is added in 2.x... sigh
-        return(s.replace("<?xml version='1.0' encoding='UTF-8'?>\n", ''))
+        return(tostring(e, encoding='unicode', method='xml'))

    def resource_from_etree(self, etree, resource_class):
        """Construct a Resource from an etree.
@ -342,7 +339,7 @@ class Sitemap(object):
            # have on element, look at attributes
            md = self.md_from_etree(md_elements[0], context=loc)
            # simple attributes that map directly to Resource object attributes
-            for att in ('capability', 'change', 'length', 'path', 'mime_type'):
+            for att in ('capability', 'change', 'datetime', 'length', 'path', 'mime_type'):
                if (att in md):
                    setattr(resource, att, md[att])
            # The ResourceSync beta spec lists md5, sha-1 and sha-256 fixity
@ -366,36 +363,30 @@ class Sitemap(object):
        Parameters:
            md_element  - etree element <rs:md>
            context     - context for error reporting
+
+        ResourceSync v1.1 adds the datetime attribute
        """
        md = {}
        # grab all understood attributes into md dict
-        for att in ('capability', 'change', 'hash', 'length', 'path', 'mime_type',
-                    'md_at', 'md_completed', 'md_from', 'md_until'):
+        for att in self.md_att_keys:
            xml_att = self._xml_att_name(att)
            val = md_element.attrib.get(xml_att, None)
            if (val is not None):
                md[att] = val
-        # capability. Allow this to be missing but do a very simple syntax
-        # check on plausible values if present
-        if ('capability' in md):
-            if (re.match(r"^[\w\-]+$", md['capability']) is None):
-                raise SitemapParseError(
-                    "Bad capability name '%s' in %s" %
-                    (capability, context))
-        # change should be one of defined values
-        if ('change' in md):
-            if (md['change'] not in ['created', 'updated', 'deleted']):
-                self.logger.warning(
-                    "Bad change attribute in <rs:md> for %s" %
-                    (context))
-        # length should be an integer
+        # capability. Allow this to be missing or a new value but warn if it
+        # isn't recognized
+        if ('capability' in md and md['capability'] not in self.capabilities):
+            self.logger.warning("Unknown capability name '%s' in %s" % (md['capability'], context))
+        # change must be one of defined values
+        if ('change' in md and md['change'] not in ['created', 'updated', 'deleted']):
+            raise SitemapParseError("Bad change attribute in <rs:md> for %s" % (context))
+        # length must be an integer
        if ('length' in md):
            try:
                md['length'] = int(md['length'])
            except ValueError as e:
-                raise SitemapParseError(
-                    "Invalid length element in <rs:md> for %s" %
-                    (context))
+                raise SitemapParseError("Invalid length element in <rs:md> for %s" %
+                                        (context))
        return(md)

    def ln_from_etree(self, ln_element, context=''):
@ -416,9 +407,8 @@ class Sitemap(object):
        # now do some checks and conversions...
        # href (MANDATORY)
        if ('href' not in ln):
-            raise SitemapParseError(
-                "Missing href in <rs:ln> in %s" %
-                (context))
+            raise SitemapParseError("Missing href in <rs:ln> in %s" %
+                                    (context))
        # rel (MANDATORY)
        if ('rel' not in ln):
            raise SitemapParseError("Missing rel in <rs:ln> in %s" % (context))
@ -427,9 +417,8 @@ class Sitemap(object):
            try:
                ln['length'] = int(ln['length'])
            except ValueError as e:
-                raise SitemapParseError(
-                    "Invalid length attribute value in <rs:ln> for %s" %
-                    (context))
+                raise SitemapParseError("Invalid length attribute value in <rs:ln> for %s" %
+                                        (context))
        # pri - priority, must be a number between 1 and 999999
        if ('pri' in ln):
            try:
@ -446,8 +435,8 @@ class Sitemap(object):

    # Metadata and link elements

-    def add_element_with_atts_to_etree(self, etree, name, atts):
-        """Add element with name and atts to etree iff there are any atts.
+    def add_element_with_atts_to_etree(self, etree, name, atts, add_return=True):
+        """Add empty element with name and atts to etree iff there are any atts.

        Parameters:
            etree - an etree object
@ -461,8 +450,17 @@ class Sitemap(object):
                xml_atts[self._xml_att_name(att)] = str(val)
        if (len(xml_atts) > 0):
            e = Element(name, xml_atts)
-            if (self.pretty_xml):
+            if (add_return and self.pretty_xml):
                e.tail = "\n"
+            # There is no real reason why the attribute keys should be sorted
+            # but ElementTree serialization up to Python 3.7 always wrote XML
+            # with attributes in sort order. This was changed in Python 3.8
+            # to be the order of addition. Adding sorting here keeps consistent
+            # behavior for all versions. See:
+            # https://docs.python.org/3/library/xml.etree.elementtree.html#element-objects
+            attribs = sorted(e.attrib.items())
+            e.attrib.clear()
+            e.attrib.update(attribs)
            etree.append(e)

    def _xml_att_name(self, att):
--- a/resync/source_description.py
+++ b/resync/source_description.py
@ -21,7 +21,7 @@ documents.
 See: http://www.openarchives.org/rs/resourcesync#SourceDesc
 """

-import collections
+import collections.abc

 from resync.resource import Resource
 from resync.resource_set import ResourceSet
@ -38,11 +38,15 @@ class SourceDescription(ListBaseWithIndex):
    to override.
    """

-    def __init__(self, resources=None, md=None, ln=None):
+    def __init__(self, resources=None, md=None, ln=None,
+                 spec_version='1.1', add_lastmod=False):
        """Initialize based on superclass ListBaseWithIndex."""
-        super(SourceDescription, self).__init__(resources=resources, md=md, ln=ln,
-                                                capability_name='description',
-                                                resources_class=ResourceSet)
+        super(SourceDescription, self).__init__(
+            resources=resources, md=md, ln=ln,
+            capability_name='description',
+            resources_class=ResourceSet,
+            spec_version=spec_version,
+            add_lastmod=add_lastmod)
        self.md['from'] = None  # usually don't want a from date

    def add(self, resource, replace=False):
@ -51,7 +55,7 @@ class SourceDescription(ListBaseWithIndex):
        Will throw a ValueError if the resource (ie. same uri) already
        exists in the capability_list, unless replace=True.
        """
-        if isinstance(resource, collections.Iterable):
+        if isinstance(resource, collections.abc.Iterable):
            for r in resource:
                self.resources.add(r, replace)
        else:
--- a/resync/url_authority.py
+++ b/resync/url_authority.py
@ -1,9 +1,6 @@
 """Determine URI authority based on DNS and paths."""

-try:  # python3
-    from urllib.parse import urlparse
-except ImportError:  # python2
-    from urlparse import urlparse
+from urllib.parse import urlparse
 import os.path


@ -70,8 +67,7 @@ class UrlAuthority(object):
            # Maybe should allow parallel for 3+ components, eg. a.example.org,
            # b.example.org
        path = os.path.dirname(s.path)
-        if (self.strict and
-                path != self.master_path and
-                not path.startswith(self.master_path)):
+        if (self.strict and path != self.master_path
+                and not path.startswith(self.master_path)):
            return(False)
        return(True)
--- a/resync/url_or_file_open.py
+++ b/resync/url_or_file_open.py
@ -0,0 +1,49 @@
+"""Local version of urlopen that supports local files & web URLs, plus adds auth."""
+
+import re
+import time
+from urllib.request import Request, urlopen
+
+from . import __version__
+
+
+# Global configuration settings
+NUM_REQUESTS = 0
+CONFIG = {
+    'bearer_token': None,
+    'delay': None
+}
+
+
+def set_url_or_file_open_config(key, value):
+    """Set the global config."""
+    global CONFIG
+    CONFIG[key] = value
+
+
+def url_or_file_open(uri, method=None, timeout=None):
+    """Wrapper around urlopen() to prepend file: if no scheme provided.
+
+    Can be used as a context manager because the return value from urlopen(...)
+    supports both that and straightforwrd use as simple file handle object.
+
+    If timeout is exceeded then urlopen(..) will raise a socket.timeout exception. If
+    no timeout is specified then the global default will be used.
+    """
+    if (not re.match(r'''\w+:''', uri)):
+        uri = 'file:' + uri
+    headers = {'User-Agent': 'resync/' + __version__}
+    # Do we need to send an Authorization header?
+    # FIXME - This token will be added blindy to all requests. This is insecure
+    # if the --noauth setting is used allowing requests across different domains.
+    # It would be better to have some scheme where a token is tied to a particular
+    # domain, or domain pattern.
+    if CONFIG['bearer_token'] is not None:
+        headers['Authorization'] = 'Bearer ' + CONFIG['bearer_token']
+    # Have we got a delay set? Apply only to web requests after first
+    global NUM_REQUESTS
+    if NUM_REQUESTS != 0 and CONFIG['delay'] is not None and not uri.startswith('file:'):
+        time.sleep(CONFIG['delay'])
+    NUM_REQUESTS += 1
+    maybe_timeout = {} if timeout is None else {'timeout': timeout}
+    return urlopen(Request(url=uri, headers=headers, method=method), **maybe_timeout)
--- a/resync/w3c_datetime.py
+++ b/resync/w3c_datetime.py
@ -21,10 +21,10 @@ def datetime_to_str(dt='now', no_fractions=False):
    in UTC. At preesent this code will return 6 decimal digits
    if any fraction of a second is given. It would perhaps be
    better to return only the number of decimal digits necessary,
-    up to a resultion of 1 microsecond.
+    up to a resolution of 1 microsecond.

    Special cases:
-    - Returns datetime str for now if no parameter given.
+    - Returns datetime str for 'now' if no parameter given.
    - Returns None if None is supplied.
    """
    if (dt is None):
@ -102,7 +102,7 @@ def str_to_datetime(s, context='datetime'):
    # with dt.tzinfo module but this has variation in behavior
    # between python 2.6 and 2.7... so do here for now
    m = re.match(r"(\d\d\d\d\-\d\d\-\d\dT\d\d:\d\d(:\d\d)?)(Z|([+-])"
-                 "(\d\d):(\d\d))$", s)
+                 r"(\d\d):(\d\d))$", s)
    if (m is None):
        raise ValueError("Bad datetime format (%s)" % s)
    str = m.group(1) + 'Z'
--- a/setup.py
+++ b/setup.py
@ -2,12 +2,12 @@
 from setuptools import setup, Command
 import os

-# Extract version number from resync/_version.py. Here we 
-# are very strict about the format of the version string 
-# as an extra sanity check. (Thanks for comments in 
+# Extract version number from resync/__init__.py. Here we
+# are very strict about the format of the version string
+# as an extra sanity check. (Thanks for comments in
 # http://stackoverflow.com/questions/458550/standard-way-to-embed-version-into-python-package )
 import re
-VERSIONFILE="resync/_version.py"
+VERSIONFILE = "resync/__init__.py"
 verfilestr = open(VERSIONFILE, "rt").read()
 match = re.search(r"^__version__ = '(\d\.\d.\d+(\.\d+)?)'", verfilestr, re.MULTILINE)
 if match:
@ -15,6 +15,7 @@ if match:
 else:
    raise RuntimeError("Unable to find version string in %s." % (VERSIONFILE))

+
 class Coverage(Command):
    """Class to allow coverage run from setup."""

@ -36,32 +37,30 @@ class Coverage(Command):
        os.system("coverage html")
        print("See htmlcov/index.html for details.")

+
 setup(
    name='resync',
    version=version,
    packages=['resync'],
-    scripts=['bin/resync','bin/resync-explorer'],
+    scripts=['resync-sync', 'resync-build', 'resync-explorer'],
    classifiers=["Development Status :: 4 - Beta",
                 "Intended Audience :: Developers",
                 "License :: OSI Approved :: Apache Software License",
-                 "Operating System :: OS Independent", #is this true? know Linux & OS X ok
+                 "Operating System :: OS Independent",  # is this true? know Linux & OS X ok
                 "Programming Language :: Python",
-                 "Programming Language :: Python :: 2.6",
-                 "Programming Language :: Python :: 2.7",
-                 "Programming Language :: Python :: 3.3",
-                 "Programming Language :: Python :: 3.4",
                 "Programming Language :: Python :: 3.5",
                 "Programming Language :: Python :: 3.6",
+                 "Programming Language :: Python :: 3.7",
+                 "Programming Language :: Python :: 3.8",
                 "Topic :: Internet :: WWW/HTTP",
                 "Topic :: Software Development :: Libraries :: Python Modules",
                 "Environment :: Web Environment"],
    author='Simeon Warner',
    author_email='simeon.warner@cornell.edu',
    description='ResourceSync library and client',
-    long_description=open('README').read(),
-    url='http://github.com/resync/resync',
+    long_description=open('README.md').read(),
+    long_description_content_type='text/markdown',
    install_requires=[
-        "requests",
        "python-dateutil>=1.5",
        "defusedxml>=0.4.1"
    ],
@ -69,6 +68,7 @@ setup(
    tests_require=[
        "testfixtures"
    ],
+    python_requires='>=3.5',
    cmdclass={
        'coverage': Coverage
    }
--- a/tests/test_capability_list.py
+++ b/tests/test_capability_list.py
@ -34,6 +34,7 @@ class TestCapabilityList(unittest.TestCase):
        caps.add_capability(rl, "http://example.org/resourcelist.xml")
        caps.md['from'] = "2013-02-07T22:39:00"
        self.assertEqual(len(caps), 1)
+        print(caps.as_xml())
        self.assertEqual(caps.as_xml(), '<?xml version=\'1.0\' encoding=\'UTF-8\'?>\n<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/"><rs:md capability="capabilitylist" from="2013-02-07T22:39:00" /><url><loc>http://example.org/resourcelist.xml</loc><rs:md capability="resourcelist" /></url></urlset>')

    def test03_multiple(self):
@ -59,8 +60,3 @@ class TestCapabilityList(unittest.TestCase):
        self.assertEqual(
            r.uri, 'http://example.org/resourcelist.xml', 'resourcelist uri')
        self.assertEqual(r.capability, 'resourcelist')
-
-if __name__ == '__main__':
-    suite = unittest.defaultTestLoader.loadTestsFromTestCase(
-        TestCapabilityList)
-    unittest.TextTestRunner().run(suite)
--- a/tests/test_change_dump.py
+++ b/tests/test_change_dump.py
@ -61,7 +61,3 @@ class TestChangeDump(unittest.TestCase):
 </urlset>'
        rd = ChangeDump()
        self.assertRaises(SitemapParseError, rd.parse, fh=io.StringIO(xml))
-
-if __name__ == '__main__':
-    suite = unittest.defaultTestLoader.loadTestsFromTestCase(TestChangeDump)
-    unittest.TextTestRunner().run(suite)
--- a/tests/test_change_list.py
+++ b/tests/test_change_list.py
@ -147,6 +147,48 @@ class TestChangeList(unittest.TestCase):
        cl = ChangeList()
        self.assertRaises(SitemapParseError, cl.parse, fh=io.StringIO(xml))

-if __name__ == '__main__':
-    suite = unittest.defaultTestLoader.loadTestsFromTestCase(TestChangeList)
-    unittest.TextTestRunner().run(suite)
+    def test11_prune_updates_before(self):
+        """Test prune_updates_before method."""
+        # v1.0
+        cl = ChangeList()
+        cl.resources.append(Resource('a', timestamp=1))
+        cl.resources.append(Resource('b', timestamp=2))
+        cl.resources.append(Resource('c', timestamp=3))
+        cl.resources.append(Resource('d', timestamp=4))
+        cl.prune_updates_before(3, spec_version='1.0')
+        self.assertEqual(len(cl.resources), 2)
+        i = iter(cl)
+        self.assertEqual(next(i).uri, 'c')
+        self.assertEqual(next(i).uri, 'd')
+        # put some more back out of order
+        cl.resources.append(Resource('a', timestamp=1))
+        cl.resources.append(Resource('b', timestamp=2))
+        cl.resources.append(Resource('e', timestamp=1000))
+        cl.prune_updates_before(3.5, spec_version='1.0')
+        self.assertEqual(len(cl.resources), 2)
+        i = iter(cl)
+        self.assertEqual(next(i).uri, 'd')
+        self.assertEqual(next(i).uri, 'e')
+        # without a timestamp
+        cl.resources.append(Resource('nt_1_0', ts_datetime=123))
+        self.assertRaises(Exception, cl.prune_updates_before, 3.5, spec_version='1.0')
+        # v1.1
+        cl = ChangeList()
+        cl.resources.append(Resource('aa', ts_datetime=1))
+        cl.resources.append(Resource('bb', ts_datetime=2))
+        cl.resources.append(Resource('cc', ts_datetime=3))
+        cl.prune_updates_before(3, spec_version='1.1')
+        self.assertEqual(len(cl.resources), 1)
+        i = iter(cl)
+        self.assertEqual(next(i).uri, 'cc')
+        # put some more back out of order
+        cl.resources.append(Resource('aa', ts_datetime=1))
+        cl.resources.append(Resource('bb', ts_datetime=2))
+        cl.resources.append(Resource('ee', ts_datetime=1000))
+        cl.prune_updates_before(3.5, spec_version='1.1')
+        self.assertEqual(len(cl.resources), 1)
+        i = iter(cl)
+        self.assertEqual(next(i).uri, 'ee')
+        # without a ts_datetime
+        cl.resources.append(Resource('nt_1_1', timestamp=456))
+        self.assertRaises(Exception, cl.prune_updates_before, 3.5, spec_version='1.1')
--- a/tests/test_client.py
+++ b/tests/test_client.py
@ -1,6 +1,4 @@
-from tests.testcase_with_tmpdir import TestCase
-from tests.capture_stdout import capture_stdout
-from tests.webserver_context import webserver
+from .testlib import TestCase, capture_stdout, webserver

 import unittest
 import re
@ -18,6 +16,7 @@ logging.basicConfig(level=logging.INFO)


 class TestClient(TestCase):
+    """Test cases for resync.client."""

    def test01_make_resource_list_empty(self):
        c = Client()
@ -77,19 +76,20 @@ class TestClient(TestCase):
        # through more cases...
        c = Client()
        dst = os.path.join(self.tmpdir, 'dst_dir1')
-        c.set_mappings(['file:tests/testdata/client/dir1', dst])
-        # audit with empty dst, should say 3 to create
-        with LogCapture() as lc:
-            c.baseline_or_audit(audit_only=True)
-            self.assertTrue(
-                re.match(r'Status:\s+NOT IN SYNC.*to create=3', lc.records[-2].msg))
-            self.assertEqual(lc.records[-1].msg, 'Completed audit')
-        # now do the sync
-        with LogCapture() as lc:
-            c.baseline_or_audit()
-            self.assertTrue(
-                re.match(r'Status:\s+SYNCED.*created=3', lc.records[-2].msg))
-            self.assertEqual(lc.records[-1].msg, 'Completed baseline sync')
+        with webserver('tests/testdata/client', 'localhost', 9999):
+            c.set_mappings(['http://localhost:9999/dir1', dst])
+            # audit with empty dst, should say 3 to create
+            with LogCapture() as lc:
+                c.baseline_or_audit(audit_only=True)
+                self.assertTrue(
+                    re.match(r'Status:\s+NOT IN SYNC.*to create=3', lc.records[-2].msg))
+                self.assertEqual(lc.records[-1].msg, 'Completed audit')
+            # now do the sync
+            with LogCapture() as lc:
+                c.baseline_or_audit()
+                self.assertTrue(
+                    re.match(r'Status:\s+SYNCED.*created=3', lc.records[-2].msg))
+                self.assertEqual(lc.records[-1].msg, 'Completed baseline sync')

    def test18_update_resource(self):
        c = Client()
@ -104,44 +104,45 @@ class TestClient(TestCase):
            self.assertTrue(
                lc.records[-1].msg.startswith('dryrun: would GET http://example.org/dir/2 '))
        c.dryrun = False
-        # get from file uri that does not exist
-        resource = Resource(uri='file:tests/testdata/i_do_not_exist')
-        self.assertRaises(ClientFatalError,
-                          c.update_resource, resource, filename)
-        # get from file uri that does not exist but with c.ignore_failures to
-        # log
-        resource = Resource(uri='file:tests/testdata/i_do_not_exist')
-        with LogCapture() as lc:
-            c.logger = logging.getLogger('resync.client')
-            c.ignore_failures = True
-            n = c.update_resource(resource, filename)
-            self.assertEqual(n, 0)
-            self.assertTrue(
-                lc.records[-1].msg.startswith('Failed to GET file:tests/testdata/i_do_not_exist '))
-        # get from file uri
-        resource = Resource(uri='file:tests/testdata/examples_from_spec/resourcesync_ex_1.xml',
-                            length=355, md5='abc',
-                            timestamp=10)
-        c.last_timestamp = 0
-        with LogCapture() as lc:
-            c.logger = logging.getLogger('resync.client')
-            n = c.update_resource(resource, filename)
-            self.assertEqual(n, 1)
-            self.assertTrue(lc.records[-1].msg.startswith('Event: {'))
-        # get from file uri with length and md5 warnings
-        resource = Resource(uri='file:tests/testdata/examples_from_spec/resourcesync_ex_1.xml',
-                            length=111, md5='abc',
-                            timestamp=10)
-        c.last_timestamp = 0
-        with LogCapture() as lc:
-            c.logger = logging.getLogger('resync.client')
-            c.hashes = set(['md5'])
-            n = c.update_resource(resource, filename)
-            self.assertEqual(n, 1)
-            self.assertTrue(lc.records[-1].msg.startswith('MD5 mismatch '))
-            self.assertTrue(
-                lc.records[-2].msg.startswith('Downloaded size for '))
-            self.assertTrue(lc.records[-3].msg.startswith('Event: {'))
+        with webserver('tests/testdata', 'localhost', 9999):
+            # get from file uri that does not exist
+            resource = Resource(uri='http://localhost:9999/i_do_not_exist')
+            self.assertRaises(ClientFatalError,
+                              c.update_resource, resource, filename)
+            # get from file uri that does not exist but with c.ignore_failures to
+            # log
+            resource = Resource(uri='http://localhost:9999/i_do_not_exist')
+            with LogCapture() as lc:
+                c.logger = logging.getLogger('resync.client')
+                c.ignore_failures = True
+                n = c.update_resource(resource, filename)
+                self.assertEqual(n, 0)
+                self.assertTrue(
+                    lc.records[-1].msg.startswith('Failed to GET http://localhost:9999/i_do_not_exist '))
+            # get from file uri
+            resource = Resource(uri='http://localhost:9999/examples_from_spec_v1_0/resourcesync_ex_1.xml',
+                                length=355, md5='abc',
+                                timestamp=10)
+            c.last_timestamp = 0
+            with LogCapture() as lc:
+                c.logger = logging.getLogger('resync.client')
+                n = c.update_resource(resource, filename)
+                self.assertEqual(n, 1)
+                self.assertTrue(lc.records[-1].msg.startswith('Event: {'))
+            # get from file uri with length and md5 warnings
+            resource = Resource(uri='http://localhost:9999/examples_from_spec_v1_0/resourcesync_ex_1.xml',
+                                length=111, md5='abc',
+                                timestamp=10)
+            c.last_timestamp = 0
+            with LogCapture() as lc:
+                c.logger = logging.getLogger('resync.client')
+                c.hashes = set(['md5'])
+                n = c.update_resource(resource, filename)
+                self.assertEqual(n, 1)
+                self.assertTrue(lc.records[-1].msg.startswith('MD5 mismatch '))
+                self.assertTrue(
+                    lc.records[-2].msg.startswith('Downloaded size for '))
+                self.assertTrue(lc.records[-3].msg.startswith('Event: {'))

    def test19_delete_resource(self):
        c = Client()
@ -198,37 +199,33 @@ class TestClient(TestCase):
        # document and identifies its type
        c = Client()
        with capture_stdout() as capturer:
-            c.sitemap_name = 'tests/testdata/examples_from_spec/resourcesync_ex_1.xml'
+            c.sitemap_name = 'tests/testdata/examples_from_spec_v1_0/resourcesync_ex_1.xml'
            c.parse_document()
-        self.assertTrue(
-            re.search(r'Parsed resourcelist document with 2 entries', capturer.result))
+        self.assertRegex(capturer.result, r'Parsed resourcelist document with 2 entries')
        with capture_stdout() as capturer:
-            c.sitemap_name = 'tests/testdata/examples_from_spec/resourcesync_ex_17.xml'
+            c.sitemap_name = 'tests/testdata/examples_from_spec_v1_0/resourcesync_ex_17.xml'
            c.parse_document()
-        self.assertTrue(
-            re.search(r'Parsed resourcedump document with 3 entries', capturer.result))
+        self.assertRegex(capturer.result, r'Parsed resourcedump document with 3 entries')
        with capture_stdout() as capturer:
-            c.sitemap_name = 'tests/testdata/examples_from_spec/resourcesync_ex_19.xml'
+            c.sitemap_name = 'tests/testdata/examples_from_spec_v1_0/resourcesync_ex_19.xml'
            c.parse_document()
-        self.assertTrue(
-            re.search(r'Parsed changelist document with 4 entries', capturer.result))
+        self.assertRegex(capturer.result, r'Parsed changelist document with 4 entries')
        with capture_stdout() as capturer:
-            c.sitemap_name = 'tests/testdata/examples_from_spec/resourcesync_ex_22.xml'
+            c.sitemap_name = 'tests/testdata/examples_from_spec_v1_0/resourcesync_ex_22.xml'
            c.parse_document()
-        self.assertTrue(
-            re.search(r'Parsed changedump document with 3 entries', capturer.result))
+        self.assertRegex(capturer.result, r'Parsed changedump document with 3 entries')
        # Document that doesn't exist
        c.sitemap_name = '/does_not_exist'
        self.assertRaises(ClientFatalError, c.parse_document)
        # and verbose with truncation...
        with capture_stdout() as capturer:
            c.verbose = True
-            c.sitemap_name = 'tests/testdata/examples_from_spec/resourcesync_ex_1.xml'
+            c.sitemap_name = 'tests/testdata/examples_from_spec_v1_0/resourcesync_ex_1.xml'
            c.max_sitemap_entries = 1
            c.parse_document()
-        self.assertTrue(re.search(r'Showing first 1 entries', capturer.result))
-        self.assertTrue(re.search(r'\[0\] ', capturer.result))
-        self.assertFalse(re.search(r'\[1\] ', capturer.result))
+        self.assertRegex(capturer.result, r'Showing first 1 entries')
+        self.assertRegex(capturer.result, r'\[0\] ')
+        self.assertNotRegex(capturer.result, r'\[1\] ')

    def test40_write_resource_list_mappings(self):
        c = Client()
@ -238,14 +235,10 @@ class TestClient(TestCase):
        with capture_stdout() as capturer:
            c.write_resource_list()
        # sys.stderr.write(capturer.result)
-        self.assertTrue(
-            re.search(r'<rs:md at="\S+" capability="resourcelist"', capturer.result))
-        self.assertTrue(
-            re.search(r'<url><loc>http://example.org/d1/file_a</loc>', capturer.result))
-        self.assertTrue(
-            re.search(r'<url><loc>http://example.org/d1/file_b</loc>', capturer.result))
-        self.assertTrue(
-            re.search(r'<url><loc>http://example.org/d2/file_x</loc>', capturer.result))
+        self.assertRegex(capturer.result, r'<rs:md at="\S+" capability="resourcelist"')
+        self.assertRegex(capturer.result, r'<url><loc>http://example.org/d1/file_a</loc>')
+        self.assertRegex(capturer.result, r'<url><loc>http://example.org/d1/file_b</loc>')
+        self.assertRegex(capturer.result, r'<url><loc>http://example.org/d2/file_x</loc>')

    def test41_write_resource_list_path(self):
        c = Client()
@ -255,21 +248,18 @@ class TestClient(TestCase):
        # included
        with capture_stdout() as capturer:
            c.write_resource_list(paths='tests/testdata/dir1', links=links)
-        self.assertTrue(
-            re.search(r'<rs:md at="\S+" capability="resourcelist"', capturer.result))
-        self.assertTrue(
-            re.search(r'<url><loc>http://example.org/dir1/file_a</loc>', capturer.result))
-        self.assertTrue(
-            re.search(r'<url><loc>http://example.org/dir1/file_b</loc>', capturer.result))
-        self.assertFalse(
-            re.search(r'<url><loc>http://example.org/dir2/file_x</loc>', capturer.result))
+        print("capturer.result: " + capturer.result)
+        self.assertRegex(capturer.result, r'<rs:md at="\S+" capability="resourcelist"')
+        self.assertRegex(capturer.result, r'<url><loc>http://example.org/dir1/file_a</loc>')
+        self.assertRegex(capturer.result, r'<url><loc>http://example.org/dir1/file_b</loc>')
+        self.assertNotRegex(capturer.result, r'<url><loc>http://example.org/dir2/file_x</loc>')
        # check link present
-        self.assertTrue(re.search(r'rel="uri_c"', capturer.result))
-        self.assertTrue(re.search(r'href="uri_d"', capturer.result))
+        self.assertRegex(capturer.result, r'rel="uri_c"')
+        self.assertRegex(capturer.result, r'href="uri_d"')
        # Travis CI does not preserve timestamps from github so test here for the file
        # size but not the datestamp
-        # self.assertTrue( re.search(r'<url><loc>http://example.org/dir1/file_a</loc><lastmod>[\w\-:]+</lastmod><rs:md length="20" /></url>', capturer.result ) )
-        # self.assertTrue( re.search(r'<url><loc>http://example.org/dir1/file_b</loc><lastmod>[\w\-:]+</lastmod><rs:md length="45" /></url>', capturer.result ) )
+        # self.assertRegex(capturer.result, r'<url><loc>http://example.org/dir1/file_a</loc><lastmod>[\w\-:]+</lastmod><rs:md length="20" /></url>')
+        # self.assertRegex(capturer.result, r'<url><loc>http://example.org/dir1/file_b</loc><lastmod>[\w\-:]+</lastmod><rs:md length="45" /></url>')
        # to file
        outfile = os.path.join(self.tmpdir, 'rl_out.xml')
        c.write_resource_list(paths='tests/testdata/dir1', outfile=outfile)
@ -290,19 +280,16 @@ class TestClient(TestCase):

    def test45_write_change_list(self):
        c = Client()
-        ex1 = 'tests/testdata/examples_from_spec/resourcesync_ex_1.xml'
+        ex1 = 'tests/testdata/examples_from_spec_v1_0/resourcesync_ex_1.xml'
        with capture_stdout() as capturer:
            c.write_change_list(ref_sitemap=ex1, newref_sitemap=ex1)
-        self.assertTrue(
-            re.search(r'<rs:md capability="changelist"', capturer.result))
+        self.assertRegex(capturer.result, r'<rs:md capability="changelist"')
        # compare ex1 with testdata on disk
        c.set_mappings(['http://example.org/', 'tests/testdata/'])
        with capture_stdout() as capturer:
            c.write_change_list(ref_sitemap=ex1, paths='tests/testdata/dir1')
-        self.assertTrue(
-            re.search(r'<rs:md capability="changelist"', capturer.result))
-        self.assertTrue(re.search(
-            r'<url><loc>http://example.com/res1</loc><rs:md change="deleted" /></url>', capturer.result))
+        self.assertRegex(capturer.result, r'<rs:md capability="changelist"')
+        self.assertRegex(capturer.result, r'<url><loc>http://example.com/res1</loc><rs:md change="deleted" /></url>')
        # to file
        outfile = os.path.join(self.tmpdir, 'cl_out.xml')
        c.write_change_list(
@ -315,13 +302,10 @@ class TestClient(TestCase):
        # simple case to STDOUT
        with capture_stdout() as capturer:
            c.write_capability_list(caps)
-        self.assertTrue(re.search(r'<urlset ', capturer.result))
-        self.assertTrue(
-            re.search(r'<rs:md capability="capabilitylist" />', capturer.result))
-        self.assertTrue(
-            re.search(r'<url><loc>uri_a</loc><rs:md capability="a"', capturer.result))
-        self.assertTrue(
-            re.search(r'<url><loc>uri_b</loc><rs:md capability="b"', capturer.result))
+        self.assertRegex(capturer.result, r'<urlset ')
+        self.assertRegex(capturer.result, r'<rs:md capability="capabilitylist" />')
+        self.assertRegex(capturer.result, r'<url><loc>uri_a</loc><rs:md capability="a"')
+        self.assertRegex(capturer.result, r'<url><loc>uri_b</loc><rs:md capability="b"')
        # to file (just check that something is written)
        outfile = os.path.join(self.tmpdir, 'caps_out.xml')
        c.write_capability_list(capabilities=caps, outfile=outfile)
@ -332,18 +316,15 @@ class TestClient(TestCase):
        # simple case to STDOUT
        with capture_stdout() as capturer:
            c.write_source_description(['a', 'b', 'c'])
-        self.assertTrue(re.search(r'<urlset ', capturer.result))
-        self.assertTrue(
-            re.search(r'<rs:md capability="description" />', capturer.result))
-        self.assertTrue(re.search(
-            r'<url><loc>a</loc><rs:md capability="capabilitylist" /></url>', capturer.result))
-        self.assertTrue(re.search(
-            r'<url><loc>b</loc><rs:md capability="capabilitylist" /></url>', capturer.result))
+        self.assertRegex(capturer.result, r'<urlset ')
+        self.assertRegex(capturer.result, r'<rs:md capability="description" />')
+        self.assertRegex(capturer.result, r'<url><loc>a</loc><rs:md capability="capabilitylist" /></url>')
+        self.assertRegex(capturer.result, r'<url><loc>b</loc><rs:md capability="capabilitylist" /></url>')
        # more complex case to STDOUT
        with capture_stdout() as capturer:
            c.write_source_description(
                capability_lists=['http://a.b/'], links=[{'rel': 'c', 'href': 'd'}])
-        self.assertTrue(re.search(r'http://a.b/', capturer.result))
+        self.assertRegex(capturer.result, r'http://a.b/')
        # to file (just check that something is written)
        outfile = os.path.join(self.tmpdir, 'sd_out.xml')
        c.write_source_description(capability_lists=[
@ -357,31 +338,30 @@ class TestClient(TestCase):
        # with dump file
        with capture_stdout() as capturer:
            c.write_dump_if_requested(ChangeList(), '/tmp/a_file')
-        self.assertTrue(re.search(r'FIXME', capturer.result))
+        self.assertRegex(capturer.result, r'FIXME')

    def test49_read_reference_resource_list(self):
        c = Client()
        with capture_stdout() as capturer:
            rl = c.read_reference_resource_list(
-                'tests/testdata/examples_from_spec/resourcesync_ex_1.xml')
+                'tests/testdata/examples_from_spec_v1_0/resourcesync_ex_1.xml')
        self.assertEqual(len(rl), 2)
        self.assertEqual('', capturer.result)
        c.verbose = True
        with capture_stdout() as capturer:
            rl = c.read_reference_resource_list(
-                'tests/testdata/examples_from_spec/resourcesync_ex_1.xml')
+                'tests/testdata/examples_from_spec_v1_0/resourcesync_ex_1.xml')
        self.assertEqual(len(rl), 2)
-        self.assertTrue(re.search(r'http://example.com/res2', capturer.result))
+        self.assertRegex(capturer.result, r'http://example.com/res2')
        c.verbose = True
        c.max_sitemap_entries = 1
        with capture_stdout() as capturer:
            rl = c.read_reference_resource_list(
-                'tests/testdata/examples_from_spec/resourcesync_ex_1.xml')
+                'tests/testdata/examples_from_spec_v1_0/resourcesync_ex_1.xml')
        self.assertEqual(len(rl), 2)
-        self.assertTrue(re.search(r'http://example.com/res1', capturer.result))
-        self.assertTrue(re.search(r'Showing first 1 entries', capturer.result))
-        self.assertFalse(
-            re.search(r'http://example.com/res2', capturer.result))
+        self.assertRegex(capturer.result, r'http://example.com/res1')
+        self.assertRegex(capturer.result, r'Showing first 1 entries')
+        self.assertNotRegex(capturer.result, r'http://example.com/res2')

    def test50_log_status(self):
        c = Client()
@ -412,7 +392,3 @@ class TestClient(TestCase):
            c.log_status(in_sync=False)
            self.assertEqual(lc.records[-1].msg,
                             'Status:          SYNCED (created=0, updated=0, deleted=0)')
-
-if __name__ == '__main__':
-    suite = unittest.TestLoader().loadTestsFromTestCase(TestClient)
-    unittest.TextTestRunner(verbosity=2).run(suite)
--- a/tests/test_client_state.py
+++ b/tests/test_client_state.py
@ -1,6 +1,6 @@
-from tests.testcase_with_tmpdir import TestCase
+from .testlib import TestCase
+
 import os.path
-import unittest
 from resync.client_state import ClientState


@ -19,7 +19,3 @@ class TestClientState(TestCase):
        self.assertEqual(cs.get_state(site), 456)
        cs.set_state(site)
        self.assertEqual(cs.get_state(site), None)
-
-if __name__ == '__main__':
-    suite = unittest.defaultTestLoader.loadTestsFromTestCase(TestClientUtils)
-    unittest.TextTestRunner(verbosity=2).run(suite)
--- a/tests/test_client_utils.py
+++ b/tests/test_client_utils.py
@ -1,4 +1,4 @@
-from tests.testcase_with_tmpdir import TestCase
+from .testlib import TestCase

 import logging
 import os.path
@ -75,7 +75,3 @@ class TestClientUtils(TestCase):
    def test06_parse_capability_lists(self):
        # Input string of the form: uri,uri
        self.assertEqual(parse_capability_lists('a,b'), ['a', 'b'])
-
-if __name__ == '__main__':
-    suite = unittest.defaultTestLoader.loadTestsFromTestCase(TestClientUtils)
-    unittest.TextTestRunner(verbosity=2).run(suite)
--- a/tests/test_dump.py
+++ b/tests/test_dump.py
@ -1,8 +1,6 @@
-from tests.testcase_with_tmpdir import TestCase
+from .testlib import TestCase

 import os.path
-import unittest
-import sys
 import zipfile

 from resync.dump import Dump, DumpError
@ -164,7 +162,3 @@ class TestDump(TestCase):
        d = Dump(rl)
        self.assertTrue(d.check_files(check_length=False))
        self.assertRaises(DumpError, d.check_files)
-
-if __name__ == '__main__':
-    suite = unittest.defaultTestLoader.loadTestsFromTestCase(TestDump)
-    unittest.TextTestRunner(verbosity=2).run(suite)
--- a/tests/test_examples_from_spec_v1_0.py
+++ b/tests/test_examples_from_spec_v1_0.py
@ -2,7 +2,7 @@
 """
 import re
 import sys
-import unittest
+
 from resync.resource import Resource
 from resync.source_description import SourceDescription
 from resync.capability_list import CapabilityList
@ -15,8 +15,13 @@ from resync.change_dump_manifest import ChangeDumpManifest
 from resync.archives import ResourceListArchive, ResourceDumpArchive, ChangeListArchive, ChangeDumpArchive
 from resync.sitemap import Sitemap

+from .testlib.testcase_with_xml_comparison import TestCase

-class TestExamplesFromSpec(unittest.TestCase):
+
+class TestExamplesFromSpec(TestCase):
+
+    def _ex_path(self, ex):
+        return 'tests/testdata/examples_from_spec_v1_0/%s.xml' % (ex)

    def test_all_simple_read(self):
        """Just try to read each one"""
@ -36,13 +41,13 @@ class TestExamplesFromSpec(unittest.TestCase):
                   'resourcesync_ex_28', 'resourcesync_ex_29', 'resourcesync_ex_30',
                   'resourcesync_ex_31', 'resourcesync_ex_32', 'resourcesync_ex_33'):
            s = Sitemap()
-            fh = self._open_ex(ex)
-            si = s.parse_xml(fh=fh)
+            with open(self._ex_path(ex), 'r') as fh:
+                si = s.parse_xml(fh=fh)

    def test_ex_01(self):
        """resourcesync_ex_1 is a simple resource_list with 2 resources, no metadata"""
-        rl = ResourceList()
-        rl.parse(uri='tests/testdata/examples_from_spec/resourcesync_ex_1.xml')
+        rl = ResourceList(spec_version='1.0')
+        rl.parse(uri='tests/testdata/examples_from_spec_v1_0/resourcesync_ex_1.xml')
        self.assertEqual(rl.capability, 'resourcelist')
        self.assertEqual(len(rl.resources), 2, '2 resources')
        sms = sorted(rl.uris())
@ -52,8 +57,8 @@ class TestExamplesFromSpec(unittest.TestCase):

    def test_ex_02(self):
        """resourcesync_ex_2 is a simple resource_list with 2 resources, some metadata"""
-        rl = ResourceList()
-        rl.parse(uri='tests/testdata/examples_from_spec/resourcesync_ex_2.xml')
+        rl = ResourceList(spec_version='1.0')
+        rl.parse(uri='tests/testdata/examples_from_spec_v1_0/resourcesync_ex_2.xml')
        self.assertEqual(len(rl.resources), 2, '2 resources')
        sms = sorted(rl.uris())
        self.assertEqual(
@ -69,12 +74,11 @@ class TestExamplesFromSpec(unittest.TestCase):

    def test_ex_03(self):
        """resourcesync_ex_3 is a simple change_list with 2 resources"""
-        cl = ChangeList()
-        cl.parse('tests/testdata/examples_from_spec/resourcesync_ex_3.xml')
+        cl = ChangeList(spec_version='1.0')
+        cl.parse('tests/testdata/examples_from_spec_v1_0/resourcesync_ex_3.xml')
        self.assertEqual(len(cl.resources), 2, '2 resources')
        sms = sorted(cl.uris())
-        self.assertEqual(
-            sms, ['http://example.com/res2.pdf', 'http://example.com/res3.tiff'])
+        self.assertEqual(sms, ['http://example.com/res2.pdf', 'http://example.com/res3.tiff'])
        self.assertEqual(cl.resources[0].lastmod, '2013-01-02T13:00:00Z')
        self.assertEqual(cl.resources[1].lastmod, '2013-01-02T18:00:00Z')
        self.assertEqual(cl.resources[0].change, 'updated')
@ -82,8 +86,8 @@ class TestExamplesFromSpec(unittest.TestCase):

    def test_ex_04(self):
        """resourcesync_ex_4 is a simple resource dump with one ZIP listed"""
-        rd = ResourceDump()
-        rd.parse('tests/testdata/examples_from_spec/resourcesync_ex_4.xml')
+        rd = ResourceDump(spec_version='1.0')
+        rd.parse('tests/testdata/examples_from_spec_v1_0/resourcesync_ex_4.xml')
        self.assertEqual(len(rd.resources), 1, '1 resources')
        self.assertTrue('http://example.com/resourcedump.zip' in rd.resources)
        self.assertEqual(rd.resources[
@ -91,8 +95,8 @@ class TestExamplesFromSpec(unittest.TestCase):

    def test_ex_05(self):
        """resourcesync_ex_5 is a simple resource dump manifest with two files listed"""
-        rdm = ResourceDumpManifest()
-        rdm.parse('tests/testdata/examples_from_spec/resourcesync_ex_5.xml')
+        rdm = ResourceDumpManifest(spec_version='1.0')
+        rdm.parse('tests/testdata/examples_from_spec_v1_0/resourcesync_ex_5.xml')
        self.assertEqual(len(rdm.resources), 2, '2 resources')
        sms = sorted(rdm.uris())
        self.assertEqual(
@ -112,8 +116,8 @@ class TestExamplesFromSpec(unittest.TestCase):

    def test_ex_06(self):
        """resourcesync_ex_6 is a simple capability list with three capabilities"""
-        capl = CapabilityList()
-        capl.parse('tests/testdata/examples_from_spec/resourcesync_ex_6.xml')
+        capl = CapabilityList(spec_version='1.0')
+        capl.parse('tests/testdata/examples_from_spec_v1_0/resourcesync_ex_6.xml')
        self.assertEqual(len(capl.resources), 3, '3 capabilities')
        # What capabilities are present?
        self.assertTrue(capl.has_capability('resourcelist'))
@ -132,8 +136,8 @@ class TestExamplesFromSpec(unittest.TestCase):

    def text_ex_07(self):
        """resourcesync_ex_7 is a source description that list a single Capability List"""
-        sd = SourceDescription()
-        sd.read(uri='tests/testdata/examples_from_spec/resourcesync_ex_7.xml')
+        sd = SourceDescription(spec_version='1.0')
+        sd.read(uri='tests/testdata/examples_from_spec_v1_0/resourcesync_ex_7.xml')
        self.assertEqual(len(sd.resources), 1, '1 capability list')
        cl = sd.resources[0]
        self.assertEqual(
@ -144,9 +148,9 @@ class TestExamplesFromSpec(unittest.TestCase):

    def test_ex_08(self):
        """resourcesync_ex_8 is a simple Resource List Index with 2 Resource Lists"""
-        rl = ResourceList()
+        rl = ResourceList(spec_version='1.0')
        rl.read(
-            uri='tests/testdata/examples_from_spec/resourcesync_ex_8.xml', index_only=True)
+            uri='tests/testdata/examples_from_spec_v1_0/resourcesync_ex_8.xml', index_only=True)
        self.assertEqual(rl.capability, 'resourcelist')
        self.assertEqual(rl.md_at, '2013-01-03T09:00:00Z')
        self.assertEqual(len(rl.resources), 2, '2 resources')
@ -158,8 +162,8 @@ class TestExamplesFromSpec(unittest.TestCase):

    def test_ex_12(self):
        """resourcesync_ex_12 is a Source Description that talks about 3 sets of resources"""
-        sd = SourceDescription()
-        sd.read(uri='tests/testdata/examples_from_spec/resourcesync_ex_12.xml')
+        sd = SourceDescription(spec_version='1.0')
+        sd.read(uri='tests/testdata/examples_from_spec_v1_0/resourcesync_ex_12.xml')
        self.assertEqual(len(sd), 3)
        self.assertEqual(sd.uris(), ['http://example.com/capabilitylist1.xml',
                                     'http://example.com/capabilitylist2.xml',
@ -169,21 +173,18 @@ class TestExamplesFromSpec(unittest.TestCase):
        self.assertEqual(
            cl1.describedby, 'http://example.com/info_about_set1_of_resources.xml')

-
-# BUILD EXAMPLES ---------------------------
-
    def test_build_ex_01(self):
-        """Simple Resource List document """
-        rl = ResourceList()
+        """Simple Resource List document"""
+        rl = ResourceList(spec_version='1.0')
        rl.md_at = '2013-01-03T09:00:00Z'
        rl.add(Resource('http://example.com/res1'))
        rl.add(Resource('http://example.com/res2'))
-        ex_xml = self._open_ex('resourcesync_ex_1').read()
+        ex_xml = self._read_ex('resourcesync_ex_1')
        self._assert_xml_equal(rl.as_xml(), ex_xml)

    def test_build_ex_02(self):
-        """Slightly more complex Resource List document """
-        rl = ResourceList()
+        """Slightly more complex Resource List document"""
+        rl = ResourceList(spec_version='1.0')
        rl.md_at = '2013-01-03T09:00:00Z'
        rl.add(Resource(uri='http://example.com/res1',
                        lastmod='2013-01-02T13:00:00Z',
@ -193,12 +194,12 @@ class TestExamplesFromSpec(unittest.TestCase):
                      md5='1e0d5cb8ef6ba40c99b14c0237be735e')
        r2.link_set(rel="duplicate", href="http://mirror.example.com/res2")
        rl.add(r2)
-        ex_xml = self._open_ex('resourcesync_ex_2').read()
+        ex_xml = self._read_ex('resourcesync_ex_2')
        self._assert_xml_equal(rl.as_xml(), ex_xml)

    def test_build_ex_03(self):
        """Simple Change List document """
-        cl = ChangeList()
+        cl = ChangeList(spec_version='1.0')
        cl.md_from = '2013-01-02T00:00:00Z'
        cl.md_until = '2013-01-03T00:00:00Z'
        cl.add(Resource(uri='http://example.com/res2.pdf',
@ -207,21 +208,21 @@ class TestExamplesFromSpec(unittest.TestCase):
        cl.add(Resource(uri='http://example.com/res3.tiff',
                        lastmod='2013-01-02T18:00:00Z',
                        change='deleted'))
-        ex_xml = self._open_ex('resourcesync_ex_3').read()
+        ex_xml = self._read_ex('resourcesync_ex_3')
        self._assert_xml_equal(cl.as_xml(), ex_xml)

    def test_build_ex_04(self):
        """Simple Resource Dump document """
-        rd = ResourceDump()
+        rd = ResourceDump(spec_version='1.0')
        rd.md_at = '2013-01-03T09:00:00Z'
        rd.add(Resource(uri='http://example.com/resourcedump.zip',
                        lastmod='2013-01-03T09:00:00Z'))
-        ex_xml = self._open_ex('resourcesync_ex_4').read()
+        ex_xml = self._read_ex('resourcesync_ex_4')
        self._assert_xml_equal(rd.as_xml(), ex_xml)

    def test_build_ex_05(self):
        """Simple Resource Dump Manifest document """
-        rdm = ResourceDumpManifest()
+        rdm = ResourceDumpManifest(spec_version='1.0')
        rdm.md_at = '2013-01-03T09:00:00Z'
        rdm.add(Resource(uri='http://example.com/res1',
                         lastmod='2013-01-03T03:00:00Z',
@ -231,12 +232,12 @@ class TestExamplesFromSpec(unittest.TestCase):
                         lastmod='2013-01-03T04:00:00Z',
                         md5='1e0d5cb8ef6ba40c99b14c0237be735e',
                         path='/resources/res2'))
-        ex_xml = self._open_ex('resourcesync_ex_5').read()
+        ex_xml = self._read_ex('resourcesync_ex_5')
        self._assert_xml_equal(rdm.as_xml(), ex_xml)

    def test_build_ex_06(self):
        """Simple Capability List document """
-        cl = CapabilityList()
+        cl = CapabilityList(spec_version='1.0')
        cl.describedby = 'http://example.com/info_about_set1_of_resources.xml'
        cl.up = 'http://example.com/resourcesync_description.xml'
        cl.add_capability(
@ -245,19 +246,19 @@ class TestExamplesFromSpec(unittest.TestCase):
            uri='http://example.com/dataset1/resourcedump.xml', name='resourcedump')
        cl.add_capability(
            uri='http://example.com/dataset1/changelist.xml', name='changelist')
-        ex_xml = self._open_ex('resourcesync_ex_6').read()
+        ex_xml = self._read_ex('resourcesync_ex_6')
        self._assert_xml_equal(cl.as_xml(), ex_xml)

    def test_build_ex_07(self):
        """A Source Description document """
-        sd = SourceDescription()
+        sd = SourceDescription(spec_version='1.0')
        sd.describedby = 'http://example.com/info-about-source.xml'
        r = Resource(uri='http://example.com/dataset1/capabilitylist.xml',
                     capability='capabilitylist')
        r.link_set(rel='describedby',
                   href='http://example.com/info_about_set1_of_resources.xml')
        sd.add(r)
-        ex_xml = self._open_ex('resourcesync_ex_7').read()
+        ex_xml = self._read_ex('resourcesync_ex_7')
        self._assert_xml_equal(sd.as_xml(), ex_xml)

    def test_build_ex_08(self):
@ -268,19 +269,19 @@ class TestExamplesFromSpec(unittest.TestCase):
        writing a large Resource List in multiple files. However,
        it is possible to create manually.
        """
-        rli = ResourceList()
+        rli = ResourceList(spec_version='1.0')
        rli.sitemapindex = True
        rli.md_at = '2013-01-03T09:00:00Z'
        rli.add(Resource(uri='http://example.com/resourcelist-part1.xml'))
        rli.add(Resource(uri='http://example.com/resourcelist-part2.xml'))
-        ex_xml = self._open_ex('resourcesync_ex_8').read()
+        ex_xml = self._read_ex('resourcesync_ex_8')
        self._assert_xml_equal(rli.as_xml(), ex_xml)

    # Examples 9, 10, 11 in the spec are not XML documents

    def test_build_ex_12(self):
        """Source Description document with describedby links"""
-        sd = SourceDescription()
+        sd = SourceDescription(spec_version='1.0')
        sd.describedby = 'http://example.com/info_about_source.xml'
        cl1 = CapabilityList(uri='http://example.com/capabilitylist1.xml')
        cl1.describedby = 'http://example.com/info_about_set1_of_resources.xml'
@ -291,12 +292,12 @@ class TestExamplesFromSpec(unittest.TestCase):
        cl3 = CapabilityList(uri='http://example.com/capabilitylist3.xml')
        cl3.describedby = 'http://example.com/info_about_set3_of_resources.xml'
        sd.add_capability_list(cl3)
-        ex_xml = self._open_ex('resourcesync_ex_12').read()
+        ex_xml = self._read_ex('resourcesync_ex_12')
        self._assert_xml_equal(sd.as_xml(), ex_xml)

    def test_build_ex_13(self):
        """Capability List document with 4 entries"""
-        cl = CapabilityList()
+        cl = CapabilityList(spec_version='1.0')
        cl.describedby = 'http://example.com/info_about_set1_of_resources.xml'
        cl.up = 'http://example.com/resourcesync_description.xml'
        cl.add_capability(capability=ResourceList(
@ -307,12 +308,12 @@ class TestExamplesFromSpec(unittest.TestCase):
            uri='http://example.com/dataset1/changelist.xml'))
        cl.add_capability(capability=ChangeDump(
            uri='http://example.com/dataset1/changedump.xml'))
-        ex_xml = self._open_ex('resourcesync_ex_13').read()
+        ex_xml = self._read_ex('resourcesync_ex_13')
        self._assert_xml_equal(cl.as_xml(), ex_xml)

    def test_build_ex_14(self):
        """Resource List with 2 entries and some metadata"""
-        rl = ResourceList()
+        rl = ResourceList(spec_version='1.0')
        rl.up = 'http://example.com/dataset1/capabilitylist.xml'
        rl.md_at = "2013-01-03T09:00:00Z"
        rl.md_completed = "2013-01-03T09:01:00Z"
@ -327,13 +328,14 @@ class TestExamplesFromSpec(unittest.TestCase):
                        sha256='854f61290e2e197a11bc91063afce22e43f8ccc655237050ace766adc68dc784',
                        length=14599,
                        mime_type="application/pdf"))
-        ex_xml = self._open_ex('resourcesync_ex_14').read()
+        ex_xml = self._read_ex('resourcesync_ex_14')
        self._assert_xml_equal(rl.as_xml(), ex_xml)

    def test_build_ex_15(self):
        """Resource List Index with metadata"""
        rl = ResourceList(
-            resources_class=ResourceListOrdered)  # order in example is non-canonical
+            resources_class=ResourceListOrdered,
+            spec_version='1.0')  # order in example is non-canonical
        rl.sitemapindex = True
        rl.up = 'http://example.com/dataset1/capabilitylist.xml'
        rl.md_at = "2013-01-03T09:00:00Z"
@ -344,11 +346,11 @@ class TestExamplesFromSpec(unittest.TestCase):
                        md_at='2013-01-03T09:03:00Z'))
        rl.add(Resource(uri='http://example.com/resourcelist3.xml',
                        md_at='2013-01-03T09:07:00Z'))
-        ex_xml = self._open_ex('resourcesync_ex_15').read()
+        ex_xml = self._read_ex('resourcesync_ex_15')
        self._assert_xml_equal(rl.as_xml(), ex_xml)

    def test_build_ex_16(self):
-        rl = ResourceList()
+        rl = ResourceList(spec_version='1.0')
        rl.up = 'http://example.com/dataset1/capabilitylist.xml'
        rl.index = 'http://example.com/dataset1/resourcelist-index.xml'
        rl.md_at = "2013-01-03T09:00:00Z"
@ -362,12 +364,12 @@ class TestExamplesFromSpec(unittest.TestCase):
                        md5='4556abdf8ebdc9802ac0c6a7402c9881',
                        length=883,
                        mime_type="image/png"))
-        ex_xml = self._open_ex('resourcesync_ex_16').read()
+        ex_xml = self._read_ex('resourcesync_ex_16')
        self._assert_xml_equal(rl.as_xml(), ex_xml)

    def test_build_ex_17(self):
        """Resource Dump with 3 entries and some metadata"""
-        rd = ResourceDump()
+        rd = ResourceDump(spec_version='1.0')
        rd.up = 'http://example.com/dataset1/capabilitylist.xml'
        rd.md_at = "2013-01-03T09:00:00Z"
        rd.md_completed = "2013-01-03T09:04:00Z"
@ -398,12 +400,12 @@ class TestExamplesFromSpec(unittest.TestCase):
                    href="http://example.com/resourcedump_manifest-part3.xml",
                    mime_type="application/xml")
        rd.add(z3)
-        ex_xml = self._open_ex('resourcesync_ex_17').read()
+        ex_xml = self._read_ex('resourcesync_ex_17')
        self._assert_xml_equal(rd.as_xml(), ex_xml)

    def test_build_ex_18(self):
        """Resource Dump Manifest with 2 entries and some metadata"""
-        rdm = ResourceDumpManifest()
+        rdm = ResourceDumpManifest(spec_version='1.0')
        rdm.up = 'http://example.com/dataset1/capabilitylist.xml'
        rdm.md_at = "2013-01-03T09:00:00Z"
        rdm.md_completed = "2013-01-03T09:02:00Z"
@ -420,12 +422,12 @@ class TestExamplesFromSpec(unittest.TestCase):
                         length=14599,
                         mime_type='application/pdf',
                         path='/resources/res2'))
-        ex_xml = self._open_ex('resourcesync_ex_18').read()
+        ex_xml = self._read_ex('resourcesync_ex_18')
        self._assert_xml_equal(rdm.as_xml(), ex_xml)

    def test_build_ex_19(self):
        """Change List with 4 changes, 'open' as no until"""
-        cl = ChangeList()
+        cl = ChangeList(spec_version='1.0')
        cl.up = 'http://example.com/dataset1/capabilitylist.xml'
        cl.md_from = "2013-01-03T00:00:00Z"
        cl.add(Resource(uri='http://example.com/res1.html',
@ -440,13 +442,14 @@ class TestExamplesFromSpec(unittest.TestCase):
        cl.add(Resource(uri='http://example.com/res2.pdf',
                        lastmod='2013-01-03T21:00:00Z',
                        change='updated'))
-        ex_xml = self._open_ex('resourcesync_ex_19').read()
+        ex_xml = self._read_ex('resourcesync_ex_19')
        self._assert_xml_equal(cl.as_xml(), ex_xml)

    def test_build_ex_20(self):
        """Change List Index listing 3 Change Lists, the last one 'open'"""
        cl = ChangeListArchive(
-            resources_class=ResourceListOrdered)  # order in example is non-canonical
+            resources_class=ResourceListOrdered,   # order in example is non-canonical
+            spec_version='1.0')
        cl.sitemapindex = True
        cl.capability_name = 'changelist'
        cl.up = 'http://example.com/dataset1/capabilitylist.xml'
@ -459,12 +462,12 @@ class TestExamplesFromSpec(unittest.TestCase):
                        md_until='2013-01-03T00:00:00Z'))
        cl.add(Resource(uri='http://example.com/20130103-changelist.xml',
                        md_from='2013-01-03T00:00:00Z'))
-        ex_xml = self._open_ex('resourcesync_ex_20').read()
+        ex_xml = self._read_ex('resourcesync_ex_20')
        self._assert_xml_equal(cl.as_xml(), ex_xml)

    def test_build_ex_21(self):
        """Change List which points back to index"""
-        cl = ChangeList()
+        cl = ChangeList(spec_version='1.0')
        cl.up = 'http://example.com/dataset1/capabilitylist.xml'
        cl.index = 'http://example.com/dataset1/changelist.xml'
        cl.md_from = "2013-01-02T00:00:00Z"
@ -481,12 +484,12 @@ class TestExamplesFromSpec(unittest.TestCase):
        cl.add(Resource(uri='http://example.com/res7.html',
                        lastmod='2013-01-02T20:00:00Z',
                        change='updated'))
-        ex_xml = self._open_ex('resourcesync_ex_21').read()
+        ex_xml = self._read_ex('resourcesync_ex_21')
        self._assert_xml_equal(cl.as_xml(), ex_xml)

    def test_build_ex_22(self):
        """Change Dump with three dump files"""
-        cd = ChangeDump()
+        cd = ChangeDump(spec_version='1.0')
        cd.up = 'http://example.com/dataset1/capabilitylist.xml'
        cd.md_from = "2013-01-01T00:00:00Z"
        z1 = Resource(uri='http://example.com/20130101-changedump.zip',
@ -511,11 +514,11 @@ class TestExamplesFromSpec(unittest.TestCase):
                      mime_type="application/zip")
        z3.contents = 'http://example.com/20130103-changedump-manifest.xml'
        cd.add([z1, z2, z3])
-        ex_xml = self._open_ex('resourcesync_ex_22').read()
+        ex_xml = self._read_ex('resourcesync_ex_22')
        self._assert_xml_equal(cd.as_xml(), ex_xml)

    def test_build_ex_23(self):
-        cdm = ChangeDumpManifest()
+        cdm = ChangeDumpManifest(spec_version='1.0')
        cdm.up = "http://example.com/dataset1/capabilitylist.xml"
        cdm.md_from = "2013-01-02T00:00:00Z"
        cdm.md_until = "2013-01-03T00:00:00Z"
@ -546,7 +549,7 @@ class TestExamplesFromSpec(unittest.TestCase):
        self._assert_xml_equal_ex(cdm.as_xml(), 'resourcesync_ex_23')

    def test_build_ex_24(self):
-        cl = ChangeList()
+        cl = ChangeList(spec_version='1.0')
        cl.up = "http://example.com/dataset1/capabilitylist.xml"
        cl.md_from = "2013-01-03T00:00:00Z"
        c1 = Resource(uri="http://example.com/res1",
@ -575,7 +578,7 @@ class TestExamplesFromSpec(unittest.TestCase):
        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_24')

    def test_build_ex_25(self):
-        cl = ChangeList()
+        cl = ChangeList(spec_version='1.0')
        cl.up = "http://example.com/dataset1/capabilitylist.xml"
        cl.md_from = "2013-01-03T11:00:00Z"
        c1 = Resource(uri="http://example.com/res1",
@ -593,7 +596,7 @@ class TestExamplesFromSpec(unittest.TestCase):
        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_25')

    def test_build_ex_26(self):
-        cl = ChangeList()
+        cl = ChangeList(spec_version='1.0')
        cl.up = "http://example.com/dataset1/capabilitylist.xml"
        cl.md_from = "2013-01-03T00:00:00Z"
        c1 = Resource(uri="http://example.com/res1.html",
@ -608,7 +611,7 @@ class TestExamplesFromSpec(unittest.TestCase):
        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_26')

    def test_build_ex_27(self):
-        cl = ChangeList()
+        cl = ChangeList(spec_version='1.0')
        cl.up = "http://example.com/dataset1/capabilitylist.xml"
        cl.md_from = "2013-01-03T00:00:00Z"
        c1 = Resource(uri="http://example.com/res4",
@ -639,7 +642,7 @@ class TestExamplesFromSpec(unittest.TestCase):
        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_27')

    def test_build_ex_28(self):
-        cl = ChangeList()
+        cl = ChangeList(spec_version='1.0')
        cl.up = "http://example.com/dataset1/capabilitylist.xml"
        cl.md_from = "2013-01-03T00:00:00Z"
        c1 = Resource(uri="http://example.com/res2.pdf",
@ -668,7 +671,7 @@ class TestExamplesFromSpec(unittest.TestCase):
        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_28')

    def test_build_ex_29(self):
-        cl = ChangeList()
+        cl = ChangeList(spec_version='1.0')
        cl.up = "http://example.com/dataset1/capabilitylist.xml"
        cl.md_from = "2013-01-03T00:00:00Z"
        c1 = Resource(uri="http://example.com/res1",
@ -692,7 +695,7 @@ class TestExamplesFromSpec(unittest.TestCase):
        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_29')

    def test_build_ex_30(self):
-        cl = ChangeList()
+        cl = ChangeList(spec_version='1.0')
        cl.up = "http://example.com/dataset1/capabilitylist.xml"
        cl.md_from = "2013-01-03T00:00:00Z"
        c1 = Resource(uri="http://example.com/res1",
@ -707,7 +710,7 @@ class TestExamplesFromSpec(unittest.TestCase):
        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_30')

    def test_build_ex_31(self):
-        cl = ChangeList()
+        cl = ChangeList(spec_version='1.0')
        cl.up = "http://example.com/dataset1/capabilitylist.xml"
        cl.md_from = "2013-01-03T00:00:00Z"
        c1 = Resource(uri="http://original.example.com/res1.html",
@ -720,7 +723,7 @@ class TestExamplesFromSpec(unittest.TestCase):
        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_31')

    def test_build_ex_32(self):
-        cl = ChangeList()
+        cl = ChangeList(spec_version='1.0')
        cl.up = "http://aggregator1.example.com/dataset1/capabilitylist.xml"
        cl.md_from = "2013-01-03T11:00:00Z"
        c1 = Resource(uri="http://aggregator1.example.com/res1.html",
@ -739,7 +742,7 @@ class TestExamplesFromSpec(unittest.TestCase):
        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_32')

    def test_build_ex_33(self):
-        cl = ChangeList()
+        cl = ChangeList(spec_version='1.0')
        cl.up = "http://aggregator2.example.com/dataset1/capabilitylist.xml"
        cl.md_from = "2013-01-03T12:00:00Z"
        c1 = Resource(uri="http://aggregator2.example.com/res1.html",
@ -761,7 +764,7 @@ class TestExamplesFromSpec(unittest.TestCase):

    def test_build_archives_ex_3_1(self):
        """Resource List Archive listing 3 Resource Lists"""
-        rla = ResourceListArchive()
+        rla = ResourceListArchive(spec_version='1.0')
        rla.up = 'http://example.com/dataset1/capabilitylist.xml'
        rla.add(Resource(uri='http://example.com/resourcelist1.xml',
                         md_at='2012-11-03T09:00:00Z'))
@ -769,22 +772,22 @@ class TestExamplesFromSpec(unittest.TestCase):
                         md_at='2012-12-03T09:00:00Z'))
        rla.add(Resource(uri='http://example.com/resourcelist3.xml',
                         md_at='2013-01-03T09:00:00Z'))
-        ex_xml = self._open_ex('archives_ex_3_1').read()
+        ex_xml = self._read_ex('archives_ex_3_1')
        self._assert_xml_equal(rla.as_xml(), ex_xml)

    def test_build_archives_ex_3_2(self):
        """Resource List Archive Index listing 2 component Resource List Archives"""
-        rlai = ResourceListArchive()
+        rlai = ResourceListArchive(spec_version='1.0')
        rlai.sitemapindex = True
        rlai.up = 'http://example.com/dataset1/capabilitylist.xml'
        rlai.add(Resource(uri='http://example.com/resourcelistarchive00001.xml'))
        rlai.add(Resource(uri='http://example.com/resourcelistarchive00002.xml'))
-        ex_xml = self._open_ex('archives_ex_3_2').read()
+        ex_xml = self._read_ex('archives_ex_3_2')
        self._assert_xml_equal(rlai.as_xml(), ex_xml)

    def test_build_archives_ex_4_1(self):
        """Resource Dump Archive listing 2 Resource Dumps"""
-        rda = ResourceDumpArchive()
+        rda = ResourceDumpArchive(spec_version='1.0')
        rda.up = 'http://example.com/dataset1/capabilitylist.xml'
        rda.add(Resource(uri='http://example.com/resourcedump1.xml',
                         lastmod='2012-11-03T09:05:42Z',
@ -795,12 +798,12 @@ class TestExamplesFromSpec(unittest.TestCase):
                         lastmod='2012-12-03T09:06:12Z',
                         md_at="2012-12-03T09:00:00Z",
                         md_completed="2012-12-03T09:05:17Z"))
-        ex_xml = self._open_ex('archives_ex_4_1').read()
+        ex_xml = self._read_ex('archives_ex_4_1')
        self._assert_xml_equal(rda.as_xml(), ex_xml)

    def test_build_archives_ex_5_1(self):
        """Change List Archive listing 3 Change Lists"""
-        cla = ChangeListArchive()
+        cla = ChangeListArchive(spec_version='1.0')
        cla.up = 'http://example.com/dataset1/capabilitylist.xml'
        cla.add(Resource(uri='http://example.com/changelist1.xml',
                         md_from='2013-01-01T09:00:00Z',
@ -811,12 +814,12 @@ class TestExamplesFromSpec(unittest.TestCase):
        cla.add(Resource(uri='http://example.com/changelist3.xml',
                         md_from='2013-01-03T09:00:00Z',
                         md_until='2013-01-04T09:00:00Z'))
-        ex_xml = self._open_ex('archives_ex_5_1').read()
+        ex_xml = self._read_ex('archives_ex_5_1')
        self._assert_xml_equal(cla.as_xml(), ex_xml)

    def test_build_archives_ex_6_1(self):
        """Change Dump Archive listing 2 Change Dumps"""
-        cda = ChangeDumpArchive()
+        cda = ChangeDumpArchive(spec_version='1.0')
        cda.up = 'http://example.com/dataset1/capabilitylist.xml'
        cda.add(Resource(uri='http://example.com/changedump-w1.xml',
                         lastmod='2012-12-20T09:02:43Z',
@ -826,63 +829,5 @@ class TestExamplesFromSpec(unittest.TestCase):
                         lastmod='2012-12-27T09:01:57Z',
                         md_from="2012-01-20T09:00:00Z",
                         md_until="2013-01-27T09:00:00Z"))
-        ex_xml = self._open_ex('archives_ex_6_1').read()
+        ex_xml = self._read_ex('archives_ex_6_1')
        self._assert_xml_equal(cda.as_xml(), ex_xml)
-
-# UTILITIES FOR (APPROX) COMPARISON OF XML IN EXAMPLES AND OUTPUT
-
-    def _assert_xml_equal_ex(self, xml, ex):
-        """Compare XML supplied with XML from example file ex"""
-        ex_xml = self._open_ex(ex).read()
-        self._assert_xml_equal(xml, ex_xml)
-
-    def _assert_xml_equal(self, a, b):
-        context = "Element mismatch in\n%s\nvs\n%s\n" % (a, b)
-        aa = self._xml_massage_split(a)
-        bb = self._xml_massage_split(b)
-        ia = iter(aa)
-        ib = iter(bb)
-        try:
-            while (1):
-                self._assert_xml_elements_equal(self._xml_reorder_attributes(next(ia)),
-                                                self._xml_reorder_attributes(
-                                                    next(ib)),
-                                                context)
-        except StopIteration:
-            # all is good provided there were the same number of elements
-            pass
-        self.assertEqual(len(aa), len(bb), "Same length check\n%s" % (context))
-
-    def _assert_xml_elements_equal(self, a, b, context):
-        context = "Elements %s != %s\n%s" % (a, b, context)
-        self.assertEqual(a, b, context)
-
-    def _xml_reorder_attributes(self, xml):
-        """Manipulate string for single element with atts in alpha order
-
-        This is a bit of a fudge because of pattern matching. Should give
-        correct match for all matches, but might give matches in rare cases
-        that should not.
-        """
-        return(' '.join(sorted(xml.split(' '))))
-
-    def _xml_massage_split(self, xml):
-        """Massage XML for comparison and split by elements (on >)"""
-        xml = re.sub(r'\s+$', '', xml)
-        xml = re.sub(r'^\s+', '', xml)
-        xml = re.sub(r'\s+', ' ', xml)
-        # always one space before end of self-closing element
-        xml = re.sub(r'\s*/>', ' />', xml)
-        xml = re.sub(r'>\s+<', '><', xml)  # remove space between elements
-        # FUDGES, need to check these are OK
-        xml = re.sub(r"version='1.0'", 'version="1.0"', xml)
-        xml = re.sub(r"encoding='UTF-8'", 'encoding="UTF-8"', xml)
-        # return self.assertEqual( x, 'xx' )
-        return(xml.split('>'))
-
-    def _open_ex(self, ex):
-        return open('tests/testdata/examples_from_spec/%s.xml' % (ex), 'r')
-
-if __name__ == '__main__':
-    suite = unittest.TestLoader().loadTestsFromTestCase(TestExamplesFromSpec)
-    unittest.TextTestRunner(verbosity=2).run(suite)
--- a/tests/test_examples_from_spec_v1_1.py
+++ b/tests/test_examples_from_spec_v1_1.py
@ -0,0 +1,752 @@
+"""Tests on the examples in the specification documents
+"""
+import re
+import sys
+
+from resync.resource import Resource
+from resync.source_description import SourceDescription
+from resync.capability_list import CapabilityList
+from resync.resource_list import ResourceList, ResourceListOrdered
+from resync.resource_dump import ResourceDump
+from resync.resource_dump_manifest import ResourceDumpManifest
+from resync.change_list import ChangeList
+from resync.change_dump import ChangeDump
+from resync.change_dump_manifest import ChangeDumpManifest
+from resync.archives import ResourceListArchive, ResourceDumpArchive, ChangeListArchive, ChangeDumpArchive
+from resync.sitemap import Sitemap
+
+from .testlib.testcase_with_xml_comparison import TestCase
+
+
+class TestExamplesFromSpec(TestCase):
+
+    def _ex_path(self, ex):
+        return 'tests/testdata/examples_from_spec_v1_1/%s.xml' % (ex)
+
+    def _read_ex(self, ex):
+        with open(self._ex_path(ex), 'r') as fh:
+            content = fh.read()
+        return content
+
+    def test_all_simple_read(self):
+        """Just try to read each one"""
+        for ex in ('resourcesync_ex_1', 'resourcesync_ex_2', 'resourcesync_ex_3',
+                   'resourcesync_ex_4', 'resourcesync_ex_5', 'resourcesync_ex_6',
+                   'resourcesync_ex_7', 'resourcesync_ex_8', 'resourcesync_ex_12',
+                   'resourcesync_ex_13', 'resourcesync_ex_14', 'resourcesync_ex_15',
+                   'resourcesync_ex_16', 'resourcesync_ex_17', 'resourcesync_ex_18',
+                   'resourcesync_ex_19', 'resourcesync_ex_20', 'resourcesync_ex_21',
+                   'resourcesync_ex_22', 'resourcesync_ex_23', 'resourcesync_ex_24',
+                   'resourcesync_ex_25', 'resourcesync_ex_26', 'resourcesync_ex_27',
+                   'resourcesync_ex_28', 'resourcesync_ex_29', 'resourcesync_ex_30',
+                   'resourcesync_ex_31', 'resourcesync_ex_32', 'resourcesync_ex_33'):
+            s = Sitemap()
+            with open(self._ex_path(ex), 'r') as fh:
+                si = s.parse_xml(fh=fh)
+
+    def test_ex_01(self):
+        """resourcesync_ex_1 is a simple resource_list with 2 resources, no metadata"""
+        rl = ResourceList()
+        rl.parse(uri='tests/testdata/examples_from_spec_v1_1/resourcesync_ex_1.xml')
+        self.assertEqual(rl.capability, 'resourcelist')
+        self.assertEqual(len(rl.resources), 2, '2 resources')
+        sms = sorted(rl.uris())
+        self.assertEqual(
+            sms, ['http://example.com/res1', 'http://example.com/res2'])
+        self.assertEqual(rl.resources['http://example.com/res1'].lastmod, None)
+
+    def test_ex_02(self):
+        """resourcesync_ex_2 is a simple resource_list with 2 resources, some metadata"""
+        rl = ResourceList()
+        rl.parse(uri='tests/testdata/examples_from_spec_v1_1/resourcesync_ex_2.xml')
+        self.assertEqual(len(rl.resources), 2, '2 resources')
+        sms = sorted(rl.uris())
+        self.assertEqual(
+            sms, ['http://example.com/res1', 'http://example.com/res2'])
+        self.assertEqual(
+            rl.resources['http://example.com/res1'].lastmod, '2013-01-02T13:00:00Z')
+        self.assertEqual(
+            rl.resources['http://example.com/res2'].lastmod, '2013-01-02T14:00:00Z')
+        self.assertEqual(
+            rl.resources['http://example.com/res1'].md5, '1584abdf8ebdc9802ac0c6a7402c03b6')
+        self.assertEqual(
+            rl.resources['http://example.com/res2'].md5, '1e0d5cb8ef6ba40c99b14c0237be735e')
+
+    def test_ex_03(self):
+        """resourcesync_ex_3 is a simple change_list with 3 resources"""
+        cl = ChangeList()
+        cl.parse('tests/testdata/examples_from_spec_v1_1/resourcesync_ex_3.xml')
+        self.assertEqual(len(cl.resources), 3, '3 resources')
+        sms = sorted(cl.uris())
+        self.assertEqual(sms, ['http://example.com/res1.pdf',
+                               'http://example.com/res2.pdf', 'http://example.com/res3.tiff'])
+        self.assertEqual(cl.resources[0].lastmod, '2013-01-02T13:00:00Z')
+        self.assertEqual(cl.resources[0].change, 'updated')
+        self.assertEqual(cl.resources[0].datetime, '2013-01-02T13:00:00Z')
+        self.assertEqual(cl.resources[1].change, 'deleted')
+        self.assertEqual(cl.resources[1].datetime, '2013-01-02T14:00:00Z')
+        self.assertEqual(cl.resources[2].lastmod, '2011-01-01T00:00:00Z')
+        self.assertEqual(cl.resources[2].change, 'created')
+        self.assertEqual(cl.resources[2].datetime, '2013-01-02T15:00:00Z')
+
+    def test_ex_04(self):
+        """resourcesync_ex_4 is a simple resource dump with one ZIP listed"""
+        rd = ResourceDump(spec_version='1.1')
+        rd.parse('tests/testdata/examples_from_spec_v1_1/resourcesync_ex_4.xml')
+        self.assertEqual(len(rd.resources), 1, '1 resources')
+        self.assertTrue('http://example.com/resourcedump.zip' in rd.resources)
+        self.assertEqual(rd.resources['http://example.com/resourcedump.zip'].lastmod, None)
+
+    def test_ex_05(self):
+        """resourcesync_ex_5 is a simple resource dump manifest with two files listed"""
+        rdm = ResourceDumpManifest(spec_version='1.1')
+        rdm.parse('tests/testdata/examples_from_spec_v1_1/resourcesync_ex_5.xml')
+        self.assertEqual(len(rdm.resources), 2, '2 resources')
+        sms = sorted(rdm.uris())
+        self.assertEqual(
+            sms, ['http://example.com/res1', 'http://example.com/res2'])
+        self.assertEqual(rdm.resources['http://example.com/res1'].lastmod, None)
+        self.assertEqual(rdm.resources['http://example.com/res1'].md5, '1584abdf8ebdc9802ac0c6a7402c03b6')
+        self.assertEqual(rdm.resources['http://example.com/res1'].path, '/resources/res1')
+        self.assertEqual(rdm.resources['http://example.com/res2'].lastmod, None)
+        self.assertEqual(rdm.resources['http://example.com/res2'].md5, '1e0d5cb8ef6ba40c99b14c0237be735e')
+        self.assertEqual(rdm.resources['http://example.com/res2'].path, '/resources/res2')
+
+    def test_ex_06(self):
+        """resourcesync_ex_6 is a simple capability list with three capabilities"""
+        capl = CapabilityList()
+        capl.parse('tests/testdata/examples_from_spec_v1_1/resourcesync_ex_6.xml')
+        self.assertEqual(len(capl.resources), 3, '3 capabilities')
+        # What capabilities are present?
+        self.assertTrue(capl.has_capability('resourcelist'))
+        self.assertEqual(capl.capability_info('resourcelist').uri,
+                         'http://example.com/dataset1/resourcelist.xml')
+        self.assertTrue(capl.has_capability('resourcedump'))
+        self.assertEqual(capl.capability_info('resourcedump').uri,
+                         'http://example.com/dataset1/resourcedump.xml')
+        self.assertTrue(capl.has_capability('changelist'))
+        self.assertEqual(capl.capability_info('changelist').uri,
+                         'http://example.com/dataset1/changelist.xml')
+        # Check some that aren't
+        self.assertFalse(capl.has_capability())
+        self.assertFalse(capl.has_capability('bogus'))
+        self.assertFalse(capl.has_capability('capabilitylist'))
+
+    def text_ex_07(self):
+        """resourcesync_ex_7 is a source description that list a single Capability List"""
+        sd = SourceDescription()
+        sd.read(uri='tests/testdata/examples_from_spec_v1_1/resourcesync_ex_7.xml')
+        self.assertEqual(len(sd.resources), 1, '1 capability list')
+        cl = sd.resources[0]
+        self.assertEqual(
+            cl.uri, 'http://example.com/dataset1/capabilitylist.xml')
+        self.assertEqual(cl.capability, 'resourcelist')
+        self.assertEqual(
+            cl.describedby, 'http://example.com/info_about_set1_of_resources.xml')
+
+    def test_ex_08(self):
+        """resourcesync_ex_8 is a simple Resource List Index with 2 Resource Lists"""
+        rl = ResourceList()
+        rl.read(
+            uri='tests/testdata/examples_from_spec_v1_1/resourcesync_ex_8.xml', index_only=True)
+        self.assertEqual(rl.capability, 'resourcelist')
+        self.assertEqual(rl.md_at, '2013-01-03T09:00:00Z')
+        self.assertEqual(len(rl.resources), 2, '2 resources')
+        sms = sorted(rl.uris())
+        self.assertEqual(sms, ['http://example.com/resourcelist-part1.xml',
+                               'http://example.com/resourcelist-part2.xml'])
+
+    # Examples 9, 10, 11 in the spec are not XML documents
+
+    def test_ex_12(self):
+        """resourcesync_ex_12 is a Source Description that talks about 3 sets of resources"""
+        sd = SourceDescription()
+        sd.read(uri='tests/testdata/examples_from_spec_v1_1/resourcesync_ex_12.xml')
+        self.assertEqual(len(sd), 3)
+        self.assertEqual(sd.uris(), ['http://example.com/capabilitylist1.xml',
+                                     'http://example.com/capabilitylist2.xml',
+                                     'http://example.com/capabilitylist3.xml'])
+        cl1 = sd['http://example.com/capabilitylist1.xml']
+        self.assertEqual(cl1.capability, 'capabilitylist')
+        self.assertEqual(
+            cl1.describedby, 'http://example.com/info_about_set1_of_resources.xml')
+
+    def test_build_ex_01(self):
+        """Simple Resource List document"""
+        rl = ResourceList()
+        rl.md_at = '2013-01-03T09:00:00Z'
+        rl.add(Resource('http://example.com/res1'))
+        rl.add(Resource('http://example.com/res2'))
+        ex_xml = self._read_ex('resourcesync_ex_1')
+        self._assert_xml_equal(rl.as_xml(), ex_xml)
+
+    def test_build_ex_02(self):
+        """Slightly more complex Resource List document"""
+        rl = ResourceList()
+        rl.md_at = '2013-01-03T09:00:00Z'
+        rl.add(Resource(uri='http://example.com/res1',
+                        lastmod='2013-01-02T13:00:00Z',
+                        md5='1584abdf8ebdc9802ac0c6a7402c03b6'))
+        r2 = Resource(uri='http://example.com/res2',
+                      lastmod='2013-01-02T14:00:00Z',
+                      md5='1e0d5cb8ef6ba40c99b14c0237be735e')
+        r2.link_set(rel="duplicate", href="http://mirror.example.com/res2")
+        rl.add(r2)
+        ex_xml = self._read_ex('resourcesync_ex_2')
+        self._assert_xml_equal(rl.as_xml(), ex_xml)
+
+    def test_build_ex_03(self):
+        """Simple Change List document """
+        cl = ChangeList()
+        cl.md_from = '2013-01-02T00:00:00Z'
+        cl.md_until = '2013-01-03T00:00:00Z'
+        cl.add(Resource(uri='http://example.com/res1.pdf',
+                        lastmod='2013-01-02T13:00:00Z',
+                        change='updated',
+                        datetime='2013-01-02T13:00:00Z'))
+        cl.add(Resource(uri='http://example.com/res2.pdf',
+                        change="deleted",
+                        datetime='2013-01-02T14:00:00Z'))
+        cl.add(Resource(uri='http://example.com/res3.tiff',
+                        lastmod='2011-01-01T00:00:00Z',
+                        change='created',
+                        datetime='2013-01-02T15:00:00Z'))
+        ex_xml = self._read_ex('resourcesync_ex_3')
+        self._assert_xml_equal(cl.as_xml(), ex_xml)
+
+    def test_build_ex_04(self):
+        """Simple Resource Dump document """
+        rd = ResourceDump()
+        rd.md_at = '2013-01-03T09:00:00Z'
+        rd.add(Resource(uri='http://example.com/resourcedump.zip'))
+        ex_xml = self._read_ex('resourcesync_ex_4')
+        self._assert_xml_equal(rd.as_xml(), ex_xml)
+
+    def test_build_ex_05(self):
+        """Simple Resource Dump Manifest document """
+        rdm = ResourceDumpManifest()
+        rdm.md_at = '2013-01-03T09:00:00Z'
+        rdm.add(Resource(uri='http://example.com/res1',
+                         md5='1584abdf8ebdc9802ac0c6a7402c03b6',
+                         path='/resources/res1'))
+        rdm.add(Resource(uri='http://example.com/res2',
+                         md5='1e0d5cb8ef6ba40c99b14c0237be735e',
+                         path='/resources/res2'))
+        ex_xml = self._read_ex('resourcesync_ex_5')
+        self._assert_xml_equal(rdm.as_xml(), ex_xml)
+
+    def test_build_ex_06(self):
+        """Simple Capability List document """
+        cl = CapabilityList()
+        cl.describedby = 'http://example.com/info_about_set1_of_resources.xml'
+        cl.up = 'http://example.com/resourcesync_description.xml'
+        cl.add_capability(
+            uri='http://example.com/dataset1/resourcelist.xml', name='resourcelist')
+        cl.add_capability(
+            uri='http://example.com/dataset1/resourcedump.xml', name='resourcedump')
+        cl.add_capability(
+            uri='http://example.com/dataset1/changelist.xml', name='changelist')
+        ex_xml = self._read_ex('resourcesync_ex_6')
+        self._assert_xml_equal(cl.as_xml(), ex_xml)
+
+    def test_build_ex_07(self):
+        """A Source Description document """
+        sd = SourceDescription()
+        sd.describedby = 'http://example.com/info-about-source.xml'
+        r = Resource(uri='http://example.com/dataset1/capabilitylist.xml',
+                     capability='capabilitylist')
+        r.link_set(rel='describedby',
+                   href='http://example.com/info_about_set1_of_resources.xml')
+        sd.add(r)
+        ex_xml = self._read_ex('resourcesync_ex_7')
+        self._assert_xml_equal(sd.as_xml(), ex_xml)
+
+    def test_build_ex_08(self):
+        """Simple Resource List Index document
+
+        This is not something that would usually be created directly
+        but instead would be created as part of the process of
+        writing a large Resource List in multiple files. However,
+        it is possible to create manually.
+        """
+        rli = ResourceList()
+        rli.sitemapindex = True
+        rli.md_at = '2013-01-03T09:00:00Z'
+        rli.add(Resource(uri='http://example.com/resourcelist-part1.xml'))
+        rli.add(Resource(uri='http://example.com/resourcelist-part2.xml'))
+        ex_xml = self._read_ex('resourcesync_ex_8')
+        self._assert_xml_equal(rli.as_xml(), ex_xml)
+
+    # Examples 9, 10, 11 in the spec are not XML documents
+
+    def test_build_ex_12(self):
+        """Source Description document with describedby links"""
+        sd = SourceDescription()
+        sd.describedby = 'http://example.com/info_about_source.xml'
+        cl1 = CapabilityList(uri='http://example.com/capabilitylist1.xml')
+        cl1.describedby = 'http://example.com/info_about_set1_of_resources.xml'
+        sd.add_capability_list(cl1)
+        cl2 = CapabilityList(uri='http://example.com/capabilitylist2.xml')
+        cl2.describedby = 'http://example.com/info_about_set2_of_resources.xml'
+        sd.add_capability_list(cl2)
+        cl3 = CapabilityList(uri='http://example.com/capabilitylist3.xml')
+        cl3.describedby = 'http://example.com/info_about_set3_of_resources.xml'
+        sd.add_capability_list(cl3)
+        ex_xml = self._read_ex('resourcesync_ex_12')
+        self._assert_xml_equal(sd.as_xml(), ex_xml)
+
+    def test_build_ex_13(self):
+        """Capability List document with 4 entries"""
+        cl = CapabilityList()
+        cl.describedby = 'http://example.com/info_about_set1_of_resources.xml'
+        cl.up = 'http://example.com/resourcesync_description.xml'
+        cl.add_capability(capability=ResourceList(
+            uri='http://example.com/dataset1/resourcelist.xml'))
+        cl.add_capability(capability=ResourceDump(
+            uri='http://example.com/dataset1/resourcedump.xml'))
+        cl.add_capability(capability=ChangeList(
+            uri='http://example.com/dataset1/changelist.xml'))
+        cl.add_capability(capability=ChangeDump(
+            uri='http://example.com/dataset1/changedump.xml'))
+        ex_xml = self._read_ex('resourcesync_ex_13')
+        self._assert_xml_equal(cl.as_xml(), ex_xml)
+
+    def test_build_ex_14(self):
+        """Resource List with 2 entries and some metadata"""
+        rl = ResourceList()
+        rl.up = 'http://example.com/dataset1/capabilitylist.xml'
+        rl.md_at = "2013-01-03T09:00:00Z"
+        rl.md_completed = "2013-01-03T09:01:00Z"
+        rl.add(Resource(uri='http://example.com/res1',
+                        lastmod='2013-01-02T13:00:00Z',
+                        md5='1584abdf8ebdc9802ac0c6a7402c03b6',
+                        length=8876,
+                        mime_type="text/html"))
+        rl.add(Resource(uri='http://example.com/res2',
+                        lastmod='2013-01-02T14:00:00Z',
+                        md5='1e0d5cb8ef6ba40c99b14c0237be735e',
+                        sha256='854f61290e2e197a11bc91063afce22e43f8ccc655237050ace766adc68dc784',
+                        length=14599,
+                        mime_type="application/pdf"))
+        ex_xml = self._read_ex('resourcesync_ex_14')
+        self._assert_xml_equal(rl.as_xml(), ex_xml)
+
+    def test_build_ex_15(self):
+        """Resource List Index with metadata"""
+        rl = ResourceList(
+            resources_class=ResourceListOrdered)  # order in example is non-canonical
+        rl.sitemapindex = True
+        rl.up = 'http://example.com/dataset1/capabilitylist.xml'
+        rl.md_at = "2013-01-03T09:00:00Z"
+        rl.md_completed = "2013-01-03T09:10:00Z"
+        rl.add(Resource(uri='http://example.com/resourcelist1.xml',
+                        md_at='2013-01-03T09:00:00Z'))
+        rl.add(Resource(uri='http://example.com/resourcelist2.xml',
+                        md_at='2013-01-03T09:03:00Z'))
+        rl.add(Resource(uri='http://example.com/resourcelist3.xml',
+                        md_at='2013-01-03T09:07:00Z'))
+        ex_xml = self._read_ex('resourcesync_ex_15')
+        self._assert_xml_equal(rl.as_xml(), ex_xml)
+
+    def test_build_ex_16(self):
+        rl = ResourceList()
+        rl.up = 'http://example.com/dataset1/capabilitylist.xml'
+        rl.index = 'http://example.com/dataset1/resourcelist-index.xml'
+        rl.md_at = "2013-01-03T09:00:00Z"
+        rl.add(Resource(uri='http://example.com/res3',
+                        md5='1584abdf8ebdc9802ac0c6a7402c8753',
+                        length=4385,
+                        mime_type="application/pdf"))
+        rl.add(Resource(uri='http://example.com/res4',
+                        md5='4556abdf8ebdc9802ac0c6a7402c9881',
+                        length=883,
+                        mime_type="image/png"))
+        ex_xml = self._read_ex('resourcesync_ex_16')
+        self._assert_xml_equal(rl.as_xml(), ex_xml)
+
+    def test_build_ex_17(self):
+        """Resource Dump with 3 entries and some metadata"""
+        rd = ResourceDump()
+        rd.up = 'http://example.com/dataset1/capabilitylist.xml'
+        rd.md_at = "2013-01-03T09:00:00Z"
+        rd.md_completed = "2013-01-03T09:04:00Z"
+        z1 = Resource(uri='http://example.com/resourcedump-part1.zip',
+                      mime_type="application/zip",
+                      length=4765,
+                      md_at="2013-01-03T09:00:00Z",
+                      md_completed="2013-01-03T09:02:00Z")
+        z1.link_set(rel="contents",
+                    href="http://example.com/resourcedump_manifest-part1.xml",
+                    mime_type="application/xml")
+        rd.add(z1)
+        z2 = Resource(uri='http://example.com/resourcedump-part2.zip',
+                      mime_type="application/zip",
+                      length=9875,
+                      md_at="2013-01-03T09:01:00Z",
+                      md_completed="2013-01-03T09:03:00Z")
+        z2.link_set(rel="contents",
+                    href="http://example.com/resourcedump_manifest-part2.xml",
+                    mime_type="application/xml")
+        rd.add(z2)
+        z3 = Resource(uri='http://example.com/resourcedump-part3.zip',
+                      mime_type="application/zip",
+                      length=2298,
+                      md_at="2013-01-03T09:03:00Z",
+                      md_completed="2013-01-03T09:04:00Z")
+        z3.link_set(rel="contents",
+                    href="http://example.com/resourcedump_manifest-part3.xml",
+                    mime_type="application/xml")
+        rd.add(z3)
+        ex_xml = self._read_ex('resourcesync_ex_17')
+        self._assert_xml_equal(rd.as_xml(), ex_xml)
+
+    def test_build_ex_18(self):
+        """Resource Dump Manifest with 2 entries and some metadata"""
+        rdm = ResourceDumpManifest()
+        rdm.up = 'http://example.com/dataset1/capabilitylist.xml'
+        rdm.md_at = "2013-01-03T09:00:00Z"
+        rdm.md_completed = "2013-01-03T09:02:00Z"
+        rdm.add(Resource(uri='http://example.com/res1',
+                         lastmod='2013-01-02T13:00:00Z',
+                         md5='1584abdf8ebdc9802ac0c6a7402c03b6',
+                         length=8876,
+                         mime_type='text/html',
+                         path='/resources/res1'))
+        rdm.add(Resource(uri='http://example.com/res2',
+                         lastmod='2013-01-02T14:00:00Z',
+                         md5='1e0d5cb8ef6ba40c99b14c0237be735e',
+                         sha256='854f61290e2e197a11bc91063afce22e43f8ccc655237050ace766adc68dc784',
+                         length=14599,
+                         mime_type='application/pdf',
+                         path='/resources/res2'))
+        ex_xml = self._read_ex('resourcesync_ex_18')
+        self._assert_xml_equal(rdm.as_xml(), ex_xml)
+
+    def test_build_ex_19(self):
+        """Change List with 4 changes, 'open' as no until"""
+        cl = ChangeList()
+        cl.up = 'http://example.com/dataset1/capabilitylist.xml'
+        cl.md_from = "2013-01-03T00:00:00Z"
+        cl.add(Resource(uri='http://example.com/res1.html',
+                        lastmod='2000-01-01T01:01:00Z',
+                        change='created',
+                        datetime='2013-01-03T11:00:00Z'))
+        cl.add(Resource(uri='http://example.com/res2.pdf',
+                        lastmod='2013-01-03T13:00:00Z',
+                        change='updated',
+                        datetime='2013-01-03T13:00:00Z'))
+        cl.add(Resource(uri='http://example.com/res3.tiff',
+                        change='deleted',
+                        datetime='2013-01-03T18:00:00Z'))
+        cl.add(Resource(uri='http://example.com/res2.pdf',
+                        change='updated'))
+        ex_xml = self._read_ex('resourcesync_ex_19')
+        self._assert_xml_equal(cl.as_xml(), ex_xml)
+
+    def test_build_ex_20(self):
+        """Change List Index listing 3 Change Lists, the last one 'open'"""
+        cl = ChangeListArchive(
+            resources_class=ResourceListOrdered)  # order in example is non-canonical
+        cl.sitemapindex = True
+        cl.capability_name = 'changelist'
+        cl.up = 'http://example.com/dataset1/capabilitylist.xml'
+        cl.md_from = "2013-01-01T00:00:00Z"
+        cl.add(Resource(uri='http://example.com/20130101-changelist.xml',
+                        md_from='2013-01-01T00:00:00Z',
+                        md_until='2013-01-02T00:00:00Z'))
+        cl.add(Resource(uri='http://example.com/20130102-changelist.xml',
+                        md_from='2013-01-02T00:00:00Z',
+                        md_until='2013-01-03T00:00:00Z'))
+        cl.add(Resource(uri='http://example.com/20130103-changelist.xml',
+                        md_from='2013-01-03T00:00:00Z'))
+        ex_xml = self._read_ex('resourcesync_ex_20')
+        self._assert_xml_equal(cl.as_xml(), ex_xml)
+
+    def test_build_ex_21(self):
+        """Change List which points back to index"""
+        cl = ChangeList()
+        cl.up = 'http://example.com/dataset1/capabilitylist.xml'
+        cl.index = 'http://example.com/dataset1/changelist.xml'
+        cl.md_from = "2013-01-02T00:00:00Z"
+        cl.md_until = "2013-01-03T00:00:00Z"
+        cl.add(Resource(uri='http://example.com/res7.html',
+                        change='created',
+                        datetime='2013-01-02T12:00:00Z',))
+        cl.add(Resource(uri='http://example.com/res9.pdf',
+                        change='updated',
+                        datetime='2013-01-02T13:00:00Z',))
+        cl.add(Resource(uri='http://example.com/res5.tiff',
+                        change='deleted',
+                        datetime='2013-01-02T19:00:00Z'))
+        cl.add(Resource(uri='http://example.com/res7.html',
+                        change='updated',
+                        datetime='2013-01-02T20:00:00Z'))
+        ex_xml = self._read_ex('resourcesync_ex_21')
+        self._assert_xml_equal(cl.as_xml(), ex_xml)
+
+    def test_build_ex_22(self):
+        """Change Dump with three dump files"""
+        cd = ChangeDump()
+        cd.up = 'http://example.com/dataset1/capabilitylist.xml'
+        cd.md_from = "2013-01-01T00:00:00Z"
+        z1 = Resource(uri='http://example.com/20130101-changedump.zip',
+                      lastmod='2013-01-01T23:59:59Z',
+                      length=3109,
+                      md_from="2013-01-01T00:00:00Z",
+                      md_until="2013-01-02T00:00:00Z",
+                      mime_type="application/zip")
+        z1.contents = 'http://example.com/20130101-changedump-manifest.xml'
+        z2 = Resource(uri='http://example.com/20130102-changedump.zip',
+                      lastmod='2013-01-02T23:59:59Z',
+                      length=6629,
+                      md_from="2013-01-02T00:00:00Z",
+                      md_until="2013-01-03T00:00:00Z",
+                      mime_type="application/zip")
+        z2.contents = 'http://example.com/20130102-changedump-manifest.xml'
+        z3 = Resource(uri='http://example.com/20130103-changedump.zip',
+                      lastmod='2013-01-03T23:59:59Z',
+                      length=8124,
+                      md_from="2013-01-03T00:00:00Z",
+                      md_until="2013-01-04T00:00:00Z",
+                      mime_type="application/zip")
+        z3.contents = 'http://example.com/20130103-changedump-manifest.xml'
+        cd.add([z1, z2, z3])
+        ex_xml = self._read_ex('resourcesync_ex_22')
+        self._assert_xml_equal(cd.as_xml(), ex_xml)
+
+    def test_build_ex_23(self):
+        cdm = ChangeDumpManifest()
+        cdm.up = "http://example.com/dataset1/capabilitylist.xml"
+        cdm.md_from = "2013-01-02T00:00:00Z"
+        cdm.md_until = "2013-01-03T00:00:00Z"
+        cdm.add(Resource(uri="http://example.com/res7.html",
+                         lastmod="2013-01-02T12:00:00Z",
+                         change="created",
+                         datetime="2013-01-02T12:00:00Z",
+                         md5="1c1b0e264fa9b7e1e9aa6f9db8d6362b",
+                         length=4339,
+                         mime_type="text/html",
+                         path="/changes/res7.html"))
+        cdm.add(Resource(uri="http://example.com/res9.pdf",
+                         lastmod="2013-01-02T13:00:00Z",
+                         change="updated",
+                         datetime="2013-01-02T13:00:00Z",
+                         md5="f906610c3d4aa745cb2b986f25b37c5a",
+                         length=38297,
+                         mime_type="application/pdf",
+                         path="/changes/res9.pdf"))
+        cdm.add(Resource(uri="http://example.com/res5.tiff",
+                         change="deleted",
+                         datetime="2013-01-02T19:00:00Z"))
+        cdm.add(Resource(uri="http://example.com/res7.html",
+                         lastmod="2013-01-02T20:00:00Z",
+                         change="updated",
+                         datetime="2013-01-02T20:00:00Z",
+                         md5="0988647082c8bc51778894a48ec3b576",
+                         length="5426",  # should also take string
+                         mime_type="text/html",
+                         path="/changes/res7-v2.html"))
+        self._assert_xml_equal_ex(cdm.as_xml(), 'resourcesync_ex_23')
+
+    def test_build_ex_24(self):
+        cl = ChangeList()
+        cl.up = "http://example.com/dataset1/capabilitylist.xml"
+        cl.md_from = "2013-01-03T00:00:00Z"
+        c1 = Resource(uri="http://example.com/res1",
+                      lastmod="2013-01-03T18:00:00Z",  # Example has lastmod and no datetime
+                      change="updated",
+                      md5="1584abdf8ebdc9802ac0c6a7402c03b6",
+                      length=8876,
+                      mime_type="text/html")
+        # Resource.link_set with add or change link depending on one with
+        # the particular rel exists unless allow_duplicates=True.
+        # Resource.link_add will always add. Test both here...
+        c1.link_set(rel="duplicate",
+                    href="http://mirror1.example.com/res1",
+                    pri="1",
+                    modified="2013-01-03T18:00:00Z")
+        c1.link_set(rel="duplicate",
+                    href="http://mirror2.example.com/res1",
+                    pri="2",
+                    modified="2013-01-03T18:00:00Z",
+                    allow_duplicates=True)
+        c1.link_add(rel="duplicate",
+                    href="gsiftp://gridftp.example.com/res1",
+                    pri="3",
+                    modified="2013-01-03T18:00:23Z")
+        cl.add(c1)
+        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_24')
+
+    def test_build_ex_25(self):
+        cl = ChangeList()
+        cl.up = "http://example.com/dataset1/capabilitylist.xml"
+        cl.md_from = "2013-01-03T11:00:00Z"
+        c1 = Resource(uri="http://example.com/res1",
+                      change="updated",
+                      datetime="2013-01-03T18:00:00Z")
+        c1.link_add(rel="alternate",
+                    href="http://example.com/res1.html",
+                    type="text/html")
+        c1.link_add(rel="alternate",
+                    href="http://example.com/res1.pdf",
+                    type="application/pdf")
+        cl.add(c1)
+        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_25')
+
+    def test_build_ex_26(self):
+        cl = ChangeList()
+        cl.up = "http://example.com/dataset1/capabilitylist.xml"
+        cl.md_from = "2013-01-03T00:00:00Z"
+        c1 = Resource(uri="http://example.com/res1.html",
+                      change="updated",
+                      datetime="2013-01-03T18:00:00Z",
+                      md5="1584abdf8ebdc9802ac0c6a7402c03b6",
+                      length=8876)
+        c1.link_add(rel="canonical",
+                    href="http://example.com/res1",
+                    modified="2013-01-03T18:00:00Z")
+        cl.add(c1)
+        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_26')
+
+    def test_build_ex_27(self):
+        cl = ChangeList()
+        cl.up = "http://example.com/dataset1/capabilitylist.xml"
+        cl.md_from = "2013-01-03T00:00:00Z"
+        c1 = Resource(uri="http://example.com/res4",
+                      change="updated",
+                      sha256="f4OxZX_x_DFGFDgghgdfb6rtSx-iosjf6735432nklj",
+                      length=56778,
+                      mime_type="application/json")
+        c1.link_set(rel="http://www.openarchives.org/rs/terms/patch",
+                    href="http://example.com/res4-json-patch",
+                    hash="sha-256:y66dER_t_HWEIKpesdkeb7rtSc-ippjf9823742opld",  # FIXME - inconsistent
+                    length=73,
+                    type="application/json-patch")
+        c2 = Resource(uri="http://example.com/res5-full.tiff",
+                      change="updated",
+                      sha256="f4OxZX_x_FO5LcGBSKHWXfwtSx-j1ncoSt3SABJtkGk",
+                      length="9788456778",
+                      mime_type="image/tiff")
+        c2.link_set(rel="http://www.openarchives.org/rs/terms/patch",
+                    href="http://example.com/res5-diff",
+                    hash="sha-256:h986gT_t_87HTkjHYE76G558hY-jdfgy76t55sadJUYT",
+                    length=4533,
+                    type="application/x-tiff-diff")
+        cl.add([c1, c2])
+        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_27')
+
+    def test_build_ex_28(self):
+        cl = ChangeList()
+        cl.up = "http://example.com/dataset1/capabilitylist.xml"
+        cl.md_from = "2013-01-03T00:00:00Z"
+        c1 = Resource(uri="http://example.com/res2.pdf",
+                      change="updated",
+                      datetime="2013-01-03T18:00:00Z",
+                      md5="1584abdf8ebdc9802ac0c6a7402c03b6",
+                      length=8876,
+                      mime_type="application/pdf")
+        c1.link_set(rel="describedby",
+                    href="http://example.com/res2_dublin-core_metadata.xml",
+                    type="application/xml")
+        c2 = Resource(uri="http://example.com/res2_dublin-core_metadata.xml",
+                      change="updated",
+                      datetime="2013-01-03T19:00:00Z",
+                      mime_type="application/xml")
+        c2.link_set(rel="describes",
+                    href="http://example.com/res2.pdf",
+                    hash="md5:1584abdf8ebdc9802ac0c6a7402c03b6",
+                    length="8876",
+                    type="application/pdf")
+        c2.link_set(rel="profile",
+                    href="http://purl.org/dc/elements/1.1/")
+        cl.add([c1, c2])
+        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_28')
+
+    def test_build_ex_29(self):
+        cl = ChangeList()
+        cl.up = "http://example.com/dataset1/capabilitylist.xml"
+        cl.md_from = "2013-01-03T00:00:00Z"
+        c1 = Resource(uri="http://example.com/res1",
+                      lastmod="2013-01-03T18:00:00Z",
+                      change="updated",
+                      md5="1584abdf8ebdc9802ac0c6a7402c03b6",
+                      length=8876,
+                      mime_type="text/html")
+        c1.link_add(rel="memento",
+                    href="http://example.com/20130103070000/res1",
+                    modified="2013-01-02T18:00:00Z",
+                    hash="md5:1584abdf8ebdc9802ac0c6a7402c03b6",
+                    length="8876",
+                    type="text/html")
+        c1.link_add(rel="timegate",
+                    href="http://example.com/timegate/http://example.com/res1")
+        c1.link_add(rel="timemap",
+                    href="http://example.com/timemap/http://example.com/res1",
+                    type="application/link-format")
+        cl.add(c1)
+        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_29')
+
+    def test_build_ex_30(self):
+        cl = ChangeList()
+        cl.up = "http://example.com/dataset1/capabilitylist.xml"
+        cl.md_from = "2013-01-03T00:00:00Z"
+        c1 = Resource(uri="http://example.com/res1",
+                      change="updated",
+                      md5="1584abdf8ebdc9802ac0c6a7402c03b6",
+                      length=8876,
+                      mime_type="text/html")
+        c1.link_add(rel="collection",
+                    href="http://example.com/aggregation/0601007")
+        cl.add(c1)
+        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_30')
+
+    def test_build_ex_31(self):
+        cl = ChangeList()
+        cl.up = "http://example.com/dataset1/capabilitylist.xml"
+        cl.md_from = "2013-01-03T00:00:00Z"
+        c1 = Resource(uri="http://original.example.com/res1.html",
+                      lastmod="2013-01-03T07:00:00Z",
+                      change="updated",
+                      md5="1584abdf8ebdc9802ac0c6a7402c03b6",
+                      length=8876,
+                      mime_type="text/html")
+        cl.add(c1)
+        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_31')
+
+    def test_build_ex_32(self):
+        cl = ChangeList()
+        cl.up = "http://aggregator1.example.com/dataset1/capabilitylist.xml"
+        cl.md_from = "2013-01-03T11:00:00Z"
+        c1 = Resource(uri="http://aggregator1.example.com/res1.html",
+                      lastmod="2013-01-03T20:00:00Z",
+                      change="updated",
+                      md5="1584abdf8ebdc9802ac0c6a7402c03b6",
+                      length=8876,
+                      mime_type="text/html")
+        c1.link_add(rel="via",
+                    href="http://original.example.com/res1.html",
+                    modified="2013-01-03T07:00:00Z",
+                    hash="md5:1584abdf8ebdc9802ac0c6a7402c03b6",
+                    length="8876",
+                    type="text/html")
+        cl.add(c1)
+        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_32')
+
+    def test_build_ex_33(self):
+        cl = ChangeList()
+        cl.up = "http://aggregator2.example.com/dataset1/capabilitylist.xml"
+        cl.md_from = "2013-01-03T12:00:00Z"
+        c1 = Resource(uri="http://aggregator2.example.com/res1.html",
+                      lastmod="2013-01-04T09:00:00Z",
+                      change="updated",
+                      md5="1584abdf8ebdc9802ac0c6a7402c03b6",
+                      length=8876,
+                      mime_type="text/html")
+        c1.link_add(rel="via",
+                    href="http://original.example.com/res1.html",
+                    modified="2013-01-03T07:00:00Z",
+                    hash="md5:1584abdf8ebdc9802ac0c6a7402c03b6",
+                    length="8876",
+                    type="text/html")
+        cl.add(c1)
+        self._assert_xml_equal_ex(cl.as_xml(), 'resourcesync_ex_33')
--- a/tests/test_explorer.py
+++ b/tests/test_explorer.py
@ -1,4 +1,4 @@
-from tests.capture_stdout import capture_stdout
+from .testlib import capture_stdout

 import unittest
 import re
@ -8,7 +8,7 @@ import sys
 from resync.client import Client
 from resync.client_utils import ClientFatalError
 from resync.capability_list import CapabilityList
-from resync.explorer import Explorer, XResource, HeadResponse, ExplorerQuit
+from resync.explorer import Explorer, XResource, ExplorerQuit
 from resync.resource import Resource


@ -30,11 +30,6 @@ class TestExplorer(unittest.TestCase):
        self.assertEqual(x.acceptable_capabilities, [1, 2])
        self.assertEqual(x.checks, [3, 4])

-    def test02_head_response(self):
-        hr = HeadResponse()
-        self.assertEqual(hr.status_code, None)
-        self.assertEqual(len(hr.headers), 0)
-
    def test03_explorer_quit(self):
        eq = ExplorerQuit()
        self.assertTrue(isinstance(eq, Exception))
@ -105,13 +100,14 @@ class TestExplorer(unittest.TestCase):

    def test08_head_on_file(self):
        e = Explorer()
-        r1 = e.head_on_file('tests/testdata/does_not_exist')
-        self.assertEqual(r1.status_code, '404')
-        r2 = e.head_on_file('tests/testdata/dir1/file_a')
-        self.assertEqual(r2.status_code, '200')
+        (status_code, headers) = e.head_on_file('tests/testdata/does_not_exist')
+        self.assertEqual(status_code, '404')
+        self.assertEqual(headers, {})
+        (status_code, headers) = e.head_on_file('tests/testdata/dir1/file_a')
+        self.assertEqual(status_code, '200')
        self.assertTrue(re.match(r'^\d\d\d\d\-\d\d\-\d\d',
-                                 r2.headers['last-modified']))
-        self.assertEqual(r2.headers['content-length'], 20)
+                        headers['last-modified']))
+        self.assertEqual(headers['content-length'], 20)

    def test09_allowed_entries(self):
        e = Explorer()
@ -130,7 +126,3 @@ class TestExplorer(unittest.TestCase):
            uri = e.expand_relative_uri('https://example.org/ctx', 'abc')
        self.assertEqual(uri, 'https://example.org/abc')
        self.assertTrue(re.search(r'expanded relative URI', capturer.result))
-
-if __name__ == '__main__':
-    suite = unittest.TestLoader().loadTestsFromTestCase(TestExplorer)
-    unittest.TextTestRunner(verbosity=2).run(suite)
--- a/tests/test_hashes.py
+++ b/tests/test_hashes.py
@ -15,9 +15,9 @@ class TestUtill(unittest.TestCase):
        self.assertEqual(h.md5, None)
        self.assertEqual(h.sha1, '49844dd211aa33071a252d7cdc250a52cf39af33')
        self.assertEqual(h.sha256, '69fe6314a94800456af959d380f5d6932052478ea03d5ccac7ba0a14bd5e67c6')
+        h = resync.hashes.Hashes(['sha-256'])
+        h.compute_for_file('tests/testdata/a')
+        self.assertEqual(h.sha1, None)

    def test02_bad_type(self):
        self.assertRaises(Exception, resync.hashes.Hashes, ['md5', 'xyz'])
-
-if __name__ == '__main__':
-    unittest.main()
--- a/tests/test_list_base.py
+++ b/tests/test_list_base.py
@ -1,4 +1,4 @@
-from tests.testcase_with_tmpdir import TestCase
+from .testlib import TestCase

 import sys
 import os.path
@ -89,7 +89,3 @@ class TestListBase(TestCase):
        lb2.parse(fh=fh)
        self.assertEqual(lb2.capability, 'special')
        self.assertEqual(len(lb2), 2)
-
-if __name__ == '__main__':
-    suite = unittest.TestLoader().loadTestsFromTestCase(TestListBase)
-    unittest.TextTestRunner(verbosity=2).run(suite)
--- a/tests/test_list_base_with_index.py
+++ b/tests/test_list_base_with_index.py
@ -129,7 +129,3 @@ class TestListBaseWithIndex(unittest.TestCase):
        xml = lb.index_as_xml()
        self.assertTrue(
            re.search(r'<loc>a</loc><lastmod>2001-01-01T00:00:00Z</lastmod>', xml))
-
-if __name__ == '__main__':
-    suite = unittest.TestLoader().loadTestsFromTestCase(TestListBaseWithIndex)
-    unittest.TextTestRunner(verbosity=2).run(suite)
--- a/tests/test_mapper.py
+++ b/tests/test_mapper.py
@ -33,9 +33,8 @@ class TestMapper(unittest.TestCase):
        self.assertEqual(m.src_to_dst('http://e.org/p/'), '/tmp/q/')
        self.assertEqual(m.src_to_dst('http://e.org/p/aa'), '/tmp/q/aa')
        self.assertEqual(m.src_to_dst('http://e.org/p/aa/bb'), '/tmp/q/aa/bb')
-        self.assertEqual(m.src_to_dst(
-            'http://e.org/p/aa/bb/'), '/tmp/q/aa/bb/')
-        self.assertRaises(MapperError, m.src_to_dst, 'http://e.org/p')
+        self.assertEqual(m.src_to_dst('http://e.org/p/aa/bb/'), '/tmp/q/aa/bb/')
+        self.assertEqual(m.src_to_dst('http://e.org/p'), '/tmp/q/')
        self.assertRaises(MapperError, m.src_to_dst, 'http://e.org/pa')
        self.assertRaises(MapperError, m.src_to_dst, 'nomatch')

@ -44,7 +43,7 @@ class TestMapper(unittest.TestCase):
        self.assertEqual(m.dst_to_src('/tmp/q/'), 'http://e.org/p/')
        self.assertEqual(m.dst_to_src('/tmp/q/bb'), 'http://e.org/p/bb')
        self.assertEqual(m.dst_to_src('/tmp/q/bb/cc'), 'http://e.org/p/bb/cc')
-        self.assertRaises(MapperError, m.dst_to_src, '/tmp/q')
+        self.assertEqual(m.dst_to_src('/tmp/q'), 'http://e.org/p/')
        self.assertRaises(MapperError, m.dst_to_src, '/tmp/qa')
        self.assertRaises(MapperError, m.dst_to_src, 'nomatch')

@ -96,14 +95,15 @@ class TestMapper(unittest.TestCase):
        self.assertEqual(Mapper(['a=b', 'b=c']).default_src_uri(), 'a')
        self.assertRaises(MapperError, Mapper().default_src_uri)

-    # Tests for Map class

-    def test10_map_unsafe(self):
+class TestMap(unittest.TestCase):
+
+    def test01_map_unsafe(self):
+        """Test unsafe method."""
        self.assertFalse(Map('http://example.com/', 'path').unsafe())
        # Note the first is a URI and the second is a (silly) path in the
        # following
-        self.assertFalse(
-            Map('http://example.com/', 'http://example.com/').unsafe())
+        self.assertFalse(Map('http://example.com/', 'http://example.com/').unsafe())
        self.assertFalse(Map('a', 'b').unsafe())
        self.assertFalse(Map('path/a', 'path/b').unsafe())
        # The following are unsafe
@ -111,6 +111,28 @@ class TestMapper(unittest.TestCase):
        self.assertTrue(Map('path/a', 'path').unsafe())
        self.assertTrue(Map('path', 'path/b').unsafe())

-if __name__ == '__main__':
-    suite = unittest.defaultTestLoader.loadTestsFromTestCase(TestMapper)
-    unittest.TextTestRunner(verbosity=2).run(suite)
+    def test02_dst_to_src(self):
+        """Test dst_to_src method."""
+        m = Map('uri:a/b', 'path/c')
+        self.assertEqual(m.dst_to_src('path/c/file'), 'uri:a/b/file')
+        self.assertEqual(m.dst_to_src('path/c/file/'), 'uri:a/b/file/')
+        self.assertEqual(m.dst_to_src('path/c'), 'uri:a/b/')
+        self.assertEqual(m.dst_to_src('path/c/'), 'uri:a/b/')
+        self.assertEqual(m.dst_to_src('path/cliff'), None)
+        # Note that trailing slashes on the map elements have no effect
+        m = Map('uri:a/b/', 'path/c/')
+        self.assertEqual(m.dst_to_src('path/c'), 'uri:a/b/')
+        self.assertEqual(m.dst_to_src('path/c/'), 'uri:a/b/')
+        self.assertEqual(m.dst_to_src('path/cliff'), None)
+        # Failed maps return None
+        self.assertEqual(m.dst_to_src('c/file'), None)
+        self.assertEqual(m.dst_to_src(''), None)
+
+    def test03_src_to_dst(self):
+        """Test src_to_dst method."""
+        m = Map('uri:x/', '/tmp/gg/')
+        self.assertEqual(m.src_to_dst('uri:other'), None)
+        self.assertEqual(m.src_to_dst('uri:x'), '/tmp/gg/')
+        self.assertEqual(m.src_to_dst('uri:x/'), '/tmp/gg/')
+        self.assertEqual(m.src_to_dst('uri:x/lizard'), '/tmp/gg/lizard')
+        self.assertEqual(m.src_to_dst('uri:x/lizard/'), '/tmp/gg/lizard/')
--- a/tests/test_resource.py
+++ b/tests/test_resource.py
@ -1,3 +1,4 @@
+"""Tests for resync.resource."""
 import unittest
 import re
 from resync.resource import Resource, ChangeTypeError
@ -5,20 +6,60 @@ from resync.resource import Resource, ChangeTypeError

 class TestResource(unittest.TestCase):

-    def test01a_same(self):
+    def test01_init(self):
+        """Test __init__ method."""
+        # No uri = error
+        self.assertRaises(ValueError, Resource)
+        self.assertRaises(ValueError, Resource, timestamp=12)
+        # Create with many params
+        r1 = Resource(uri='a', timestamp=0, length=123,
+                      md5='aaa', sha1='bbb', sha256='ccc',
+                      mime_type='text/plain', change='updated',
+                      ts_datetime=1000, path='/a/b/c', ln={'x': 'y'},
+                      ts_at=2000, ts_completed=3000, ts_from=4000,
+                      ts_until=5000)
+        self.assertEqual(r1.uri, 'a')
+        self.assertEqual(r1.timestamp, 0)
+        self.assertEqual(r1.length, 123)
+        self.assertEqual(r1.md5, 'aaa')
+        self.assertEqual(r1.sha1, 'bbb')
+        self.assertEqual(r1.sha256, 'ccc')
+        self.assertEqual(r1.mime_type, 'text/plain')
+        self.assertEqual(r1.change, 'updated')
+        self.assertEqual(r1.ts_datetime, 1000)
+        self.assertEqual(r1.path, '/a/b/c')
+        self.assertEqual(r1.ln, {'x': 'y'})
+        self.assertEqual(r1.ts_at, 2000)
+        self.assertEqual(r1.ts_completed, 3000)
+        self.assertEqual(r1.ts_from, 4000)
+        self.assertEqual(r1.ts_until, 5000)
+        # Create r2 like r1
+        r2 = Resource(resource=r1)
+        self.assertEqual(r2.uri, 'a')
+        self.assertEqual(r2.timestamp, 0)
+        self.assertEqual(r2.length, 123)
+        self.assertEqual(r2.md5, 'aaa')
+        self.assertEqual(r2.sha1, 'bbb')
+        self.assertEqual(r2.sha256, 'ccc')
+        self.assertEqual(r2.mime_type, 'text/plain')
+        self.assertEqual(r2.change, 'updated')
+        self.assertEqual(r2.ts_datetime, 1000)
+        self.assertEqual(r2.path, '/a/b/c')
+        self.assertEqual(r2.ln, {'x': 'y'})
+
+    def test02_equal(self):
+        """Test equal method via = operator."""
+        # just uri
        r1 = Resource('a')
        r2 = Resource('a')
        self.assertEqual(r1, r1)
        self.assertEqual(r1, r2)
-
-    def test01b_same(self):
+        # with timestamps
        r1 = Resource(uri='a', timestamp=1234.0)
        r2 = Resource(uri='a', timestamp=1234.0)
        self.assertEqual(r1, r1)
        self.assertEqual(r1, r2)
-
-    def test01c_same(self):
-        """Same with lastmod instead of direct timestamp"""
+        # with lastmod instead of direct timestamp
        r1 = Resource('a')
        r1lm = '2012-01-01T00:00:00Z'
        r1.lastmod = r1lm
@ -40,26 +81,45 @@ class TestResource(unittest.TestCase):
            self.assertEqual(r1.timestamp, r2.timestamp, ('%s (%f) == %s (%f)' % (
                r1lm, r1.timestamp, r2lm, r2.timestamp)))
            self.assertEqual(r1, r2)
-
-    def test01d_same(self):
-        """Same with slight timestamp diff"""
+        # with slight timestamp diff
        r1 = Resource('a')
        r1.lastmod = '2012-01-02T01:02:03Z'
        r2 = Resource('a')
        r2.lastmod = '2012-01-02T01:02:03.99Z'
        self.assertNotEqual(r1.timestamp, r2.timestamp)
        self.assertEqual(r1, r2)
-
-    def test02a_diff(self):
+        # now with too much time diff
+        r1 = Resource('a', lastmod='2012-01-11')
+        r2 = Resource('a', lastmod='2012-01-22')
+        self.assertNotEqual(r1, r2)
+        # different uris
        r1 = Resource('a')
        r2 = Resource('b')
        self.assertNotEqual(r1, r2)
-
-    def test02b_diff(self):
-        r1 = Resource('a', lastmod='2012-01-11')
-        r2 = Resource('a', lastmod='2012-01-22')
-        # print 'r1 == r2 : '+str(r1==r2)
+        # same and different lengths
+        r1 = Resource('a', length=1234)
+        r2 = Resource('a', length=4321)
        self.assertNotEqual(r1, r2)
+        r2.length = r1.md5
+        self.assertEqual(r1, r2)
+        # same and different md5
+        r1.md5 = "3006f84272f2653a6cf5ec3af8f0d773"
+        r2.md5 = "3006f84272f2653a6cf5ec3af8f00000"
+        self.assertNotEqual(r1, r2)
+        r2.md5 = r1.md5
+        self.assertEqual(r1, r2)
+        # same and different sha1
+        r1.sha1 = "3be0f3af2aa4656ce38e0cef305c6eb2af4385d4"
+        r2.sha1 = "555"
+        self.assertNotEqual(r1, r2)
+        r2.sha1 = r1.sha1
+        self.assertEqual(r1, r2)
+        # same and different sha256
+        r1.sha256 = "f41094ad47ef3e93ec1021bfa40f4bf0185f1bf897533638ae5358b61713f84a"
+        r2.sha256 = "fab"
+        self.assertNotEqual(r1, r2)
+        r2.sha256 = r1.sha256
+        self.assertEqual(r1, r2)

    def test04_bad_lastmod(self):
        def setlastmod(r, v):
@ -75,10 +135,8 @@ class TestResource(unittest.TestCase):
        # Valid ISO8601 but not alloed in W3C Datetime
        self.assertRaises(ValueError, setlastmod, r, "2012-11-01T01:01:01")
        self.assertRaises(ValueError, setlastmod, r, "2012-11-01 01:01:01Z")
-        self.assertRaises(ValueError, setlastmod, r,
-                          "2012-11-01T01:01:01+0000")
-        self.assertRaises(ValueError, setlastmod, r,
-                          "2012-11-01T01:01:01-1000")
+        self.assertRaises(ValueError, setlastmod, r, "2012-11-01T01:01:01+0000")
+        self.assertRaises(ValueError, setlastmod, r, "2012-11-01T01:01:01-1000")

    def test05_lastmod_roundtrips(self):
        r = Resource('a')
@ -102,7 +160,8 @@ class TestResource(unittest.TestCase):
        r1 = Resource('def', lastmod='2012-01-01')
        self.assertEqual(repr(r1), "{'uri': 'def', 'timestamp': 1325376000}")

-    def test08_multiple_hashes(self):
+    def test08_hash(self):
+        """Test hash getter and setters."""
        r1 = Resource('abcd')
        r1.md5 = "some_md5"
        r1.sha1 = "some_sha1"
@ -110,8 +169,7 @@ class TestResource(unittest.TestCase):
        self.assertEqual(r1.md5, "some_md5")
        self.assertEqual(r1.sha1, "some_sha1")
        self.assertEqual(r1.sha256, "some_sha256")
-        self.assertEqual(
-            r1.hash, "md5:some_md5 sha-1:some_sha1 sha-256:some_sha256")
+        self.assertEqual(r1.hash, "md5:some_md5 sha-1:some_sha1 sha-256:some_sha256")
        r2 = Resource('def')
        r2.hash = "md5:ddd"
        self.assertEqual(r2.md5, 'ddd')
@ -123,6 +181,20 @@ class TestResource(unittest.TestCase):
        self.assertEqual(r2.md5, 'fff')
        self.assertEqual(r2.sha1, 'eee')
        self.assertEqual(r2.sha256, 'ggg')
+        # bogus value will reset
+        r2.hash = 11
+        self.assertEqual(r2.md5, None)
+        self.assertEqual(r2.sha1, None)
+        self.assertEqual(r2.sha256, None)
+        # string withough : will raise error
+        with self.assertRaises(ValueError):
+            r2.hash = "no-colon"
+        # dupe
+        with self.assertRaises(ValueError):
+            r2.hash = "md5:aaa md5:bbb"
+        # unknown
+        with self.assertRaises(ValueError):
+            r2.hash = "sha999:aaa"

    def test09_changetypeerror(self):
        r1 = Resource('a')
@ -131,9 +203,11 @@ class TestResource(unittest.TestCase):
        self.assertEqual(r1.change, 'deleted')
        self.assertRaises(ChangeTypeError, Resource, 'a', change="bad")
        # disable checking
+        ct = Resource.CHANGE_TYPES
        Resource.CHANGE_TYPES = False
        r1 = Resource('a', change="bad")
        self.assertEqual(r1.change, 'bad')
+        Resource.CHANGE_TYPES = ct

    def test10_md_at_roundtrips(self):
        r = Resource('a')
@ -178,6 +252,67 @@ class TestResource(unittest.TestCase):
        r = Resource(uri='tv2')
        self.assertEqual(r.mime_type, None)

-if __name__ == '__main__':
-    suite = unittest.TestLoader().loadTestsFromTestCase(TestResource)
-    unittest.TextTestRunner(verbosity=2).run(suite)
+    def test14_datetime(self):
+        # v1.1 adds a datetime attribute
+        r = Resource(uri='dt1')
+        self.assertEqual(r.datetime, None)
+        r = Resource(uri='dt2', datetime='2000-01-04T00:00:00Z')
+        self.assertEqual(r.datetime, '2000-01-04T00:00:00Z')
+
+    def test15_link(self):
+        """Test link link_href and link_set methods."""
+        r = Resource(uri='ln1')
+        self.assertEqual(r.link('up'), None)
+        self.assertEqual(r.link_href('up'), None)
+        r.link_set('up', 'uri:up')
+        self.assertEqual(r.link('up'), {'rel': 'up', 'href': 'uri:up'})
+        self.assertEqual(r.link_href('up'), 'uri:up')
+        r.link_set('down', 'uri:down')
+        self.assertEqual(r.link('down'), {'rel': 'down', 'href': 'uri:down'})
+        self.assertEqual(r.link_href('down'), 'uri:down')
+        r.link_set('up', 'uri:up2')
+        self.assertEqual(r.link('up'), {'rel': 'up', 'href': 'uri:up2'})
+        r.link_add('up', 'uri:up3')
+        self.assertEqual(r.link('up'), {'rel': 'up', 'href': 'uri:up2'})  # still get first
+        self.assertEqual(r.ln, [{'rel': 'up', 'href': 'uri:up2'},
+                                {'href': 'uri:down', 'rel': 'down'},
+                                {'rel': 'up', 'href': 'uri:up3'}])
+
+    def test16_specific_links(self):
+        """Test setters/getters for specific link types."""
+        r = Resource(uri='laughing')
+        r.describedby = 'uri:db'
+        self.assertEqual(r.describedby, 'uri:db')
+        r.up = 'uri:up'
+        self.assertEqual(r.up, 'uri:up')
+        r.index = 'uri:index'
+        self.assertEqual(r.index, 'uri:index')
+        r.contents = 'uri:ct'
+        self.assertEqual(r.contents, 'uri:ct')
+
+    def test17_basename(self):
+        """Test basename property derived from uri."""
+        r = Resource(uri='http://example.org/any/complex/path/file')
+        self.assertEqual(r.basename, 'file')
+        r.uri = 'http://example.org/any/complex/path/'
+        self.assertEqual(r.basename, '')
+        r.uri = 'http://example.org'
+        self.assertEqual(r.basename, '')
+
+    def test18_str(self):
+        """Test str method."""
+        self.assertEqual(str(Resource('uri:a')),
+                         '[ uri:a | None | None | None ]')
+        self.assertEqual(str(Resource('uri:a', timestamp=0, length=999)),
+                         '[ uri:a | 1970-01-01T00:00:00Z | 999 | None ]')
+        self.assertEqual(str(Resource('uri:a', timestamp=0, length=999, sha256='abcdef123')),
+                         '[ uri:a | 1970-01-01T00:00:00Z | 999 | abcdef123 ]')
+        self.assertEqual(str(Resource('uri:a', change='updated', ts_datetime=3661)),
+                         '[ uri:a | None | None | None | updated @ 1970-01-01T01:01:01Z ]')
+        self.assertEqual(str(Resource('uri:a', path='/a/b/c')),
+                         '[ uri:a | None | None | None | /a/b/c ]')
+
+    def test19_change_type_error(self):
+        """Test error from bad change type."""
+        cte = ChangeTypeError('unk')
+        self.assertIn('ChangeTypeError: got unk, expected one of ', str(cte))
--- a/tests/test_resource_container.py
+++ b/tests/test_resource_container.py
@ -1,6 +1,8 @@
+"""Tests for resync.resource_container."""
+
 import unittest
 from resync.resource import Resource
-from resync.resource_container import ResourceContainer
+from resync.resource_container import ResourceContainer, _str_datetime_now


 class TestResourceContainer(unittest.TestCase):
@ -8,9 +10,12 @@ class TestResourceContainer(unittest.TestCase):
    def test01_create_and_add(self):
        rc = ResourceContainer(resources=[])
        self.assertEqual(len(rc.resources), 0, "empty")
-        rc.resources.append(Resource('a', timestamp=1))
-        rc.resources.append(Resource('b', timestamp=2))
+        rc.add(Resource('a', timestamp=1))
+        rc.add(Resource('b', timestamp=2))
        self.assertEqual(len(rc.resources), 2, "two resources")
+        # Add two more
+        rc.add([Resource('c'), Resource('d')])
+        self.assertEqual(len(rc.resources), 4, "four resources")

    def test02_iter(self):
        rc = ResourceContainer(resources=[])
@ -25,27 +30,6 @@ class TestResourceContainer(unittest.TestCase):
        self.assertEqual(resources[0].uri, 'a')
        self.assertEqual(resources[3].uri, 'd')

-    def test03_prune_before(self):
-        rc = ResourceContainer()
-        rc.resources.append(Resource('a', timestamp=1))
-        rc.resources.append(Resource('b', timestamp=2))
-        rc.resources.append(Resource('c', timestamp=3))
-        rc.resources.append(Resource('d', timestamp=4))
-        rc.prune_before(3)
-        self.assertEqual(len(rc.resources), 2)
-        i = iter(rc)
-        self.assertEqual(next(i).uri, 'c')
-        self.assertEqual(next(i).uri, 'd')
-        # put some more back out of order
-        rc.resources.append(Resource('a', timestamp=1))
-        rc.resources.append(Resource('b', timestamp=2))
-        rc.resources.append(Resource('e', timestamp=1000))
-        rc.prune_before(3.5)
-        self.assertEqual(len(rc.resources), 2)
-        i = iter(rc)
-        self.assertEqual(next(i).uri, 'd')
-        self.assertEqual(next(i).uri, 'e')
-
    def test04_prune_dupes(self):
        rc = ResourceContainer()
        rc.resources.append(Resource('a', timestamp=1, change='created'))
@ -115,7 +99,36 @@ class TestResourceContainer(unittest.TestCase):
        rc.up = "up_uri"
        self.assertEqual(rc.up, "up_uri")

-if __name__ == '__main__':
-    suite = unittest.defaultTestLoader.loadTestsFromTestCase(
-        TestResourceContainer)
-    unittest.TextTestRunner().run(suite)
+    def test09_capability(self):
+        """Test capability property."""
+        rc = ResourceContainer()
+        self.assertEqual(rc.capability, None)
+        rc.capability = "fixit"
+        self.assertEqual(rc.capability, 'fixit')
+
+    def test10_link_set(self):
+        """Test link_set method."""
+        rc = ResourceContainer()
+        self.assertEqual(rc.link('alink'), None)  # non-spec rel should be supported
+        rc.link_set('alink', 'uri:l', extra1='one', extra2='two')
+        ln = rc.link('alink')
+        self.assertEqual(rc.link('alink'), {'rel': 'alink', 'href': 'uri:l',
+                                            'extra1': 'one', 'extra2': 'two'})
+        # Can update existing link
+        rc.link_set('alink', 'uri:new', extra1='1')
+        self.assertEqual(rc.link('alink'), {'rel': 'alink', 'href': 'uri:new',
+                                            'extra1': '1', 'extra2': 'two'})
+
+    def test11_index(self):
+        """Test index property."""
+        rc = ResourceContainer()
+        self.assertEqual(rc.index, None)
+        rc.index = 'uri:i'
+        self.assertEqual(rc.index, 'uri:i')
+
+    def test12_str_datetime_now(self):
+        """Test _str_datetime_now function."""
+        self.assertTrue(isinstance(_str_datetime_now(), str))
+        self.assertTrue(isinstance(_str_datetime_now('now'), str))
+        self.assertEqual(_str_datetime_now(0), '1970-01-01T00:00:00Z')
+        self.assertEqual(_str_datetime_now('2020-12-24T16:01:00Z'), '2020-12-24T16:01:00Z')
--- a/tests/test_resource_dump.py
+++ b/tests/test_resource_dump.py
@ -1,5 +1,6 @@
 """Tests for resync.resource_dump."""
-from tests.testcase_with_tmpdir import TestCase
+from .testlib import TestCase
+
 import os.path
 try:  # python2
    # Must try this first as io also exists in python2
@ -81,7 +82,3 @@ class TestResourceDump(TestCase):
        rd2.parse(dumpf)
        self.assertEqual(len(rd2), 2)
        self.assertEqual(rd2.uris(), ['aa.zip', 'bb.zip'])
-
-if __name__ == '__main__':
-    suite = unittest.defaultTestLoader.loadTestsFromTestCase(TestResourceDump)
-    unittest.TextTestRunner().run(suite)
--- a/tests/test_resource_dump_manifest.py
+++ b/tests/test_resource_dump_manifest.py
@ -63,8 +63,3 @@ class TestResourceDumpManifest(unittest.TestCase):
 </urlset>'
        rdm = ResourceDumpManifest()
        self.assertRaises(SitemapParseError, rdm.parse, fh=io.StringIO(xml))
-
-if __name__ == '__main__':
-    suite = unittest.defaultTestLoader.loadTestsFromTestCase(
-        TestResourceDumpManifest)
-    unittest.TextTestRunner().run(suite)
--- a/tests/test_resource_list.py
+++ b/tests/test_resource_list.py
@ -1,18 +1,17 @@
-import unittest
-try:  # python2
-    # Must try this first as io also exists in python2
-    # but in the wrong one!
-    import StringIO as io
-except ImportError:  # python3
-    import io
+"""Tests for resync.resource_list."""
+
+from .testlib import TestCase
+
+import os.path
+import io
 import re

 from resync.resource import Resource
-from resync.resource_list import ResourceList, ResourceListDupeError
-from resync.sitemap import SitemapParseError
+from resync.resource_list import ResourceList, ResourceListOrdered, ResourceListDupeError
+from resync.sitemap import Sitemap, SitemapParseError


-class TestResourceList(unittest.TestCase):
+class TestResourceList(TestCase):

    def test01_same(self):
        src = ResourceList()
@ -120,7 +119,8 @@ class TestResourceList(unittest.TestCase):
        r1.md5 = "aabbcc"
        self.assertEqual(i.hashes(), set(['md5']))
        r2.sha1 = "ddeeff"
-        self.assertEqual(i.hashes(), set(['md5', 'sha-1']))
+        r2.sha256 = "hhiijj"
+        self.assertEqual(i.hashes(), set(['md5', 'sha-1', 'sha-256']))

    def test08_iter(self):
        i = ResourceList()
@ -128,13 +128,26 @@ class TestResourceList(unittest.TestCase):
        i.add(Resource('b', timestamp=2))
        i.add(Resource('c', timestamp=3))
        i.add(Resource('d', timestamp=4))
-        resources = []
-        for r in i:
-            resources.append(r)
+        resources = list(i.resources)
        self.assertEqual(len(resources), 4)
        self.assertEqual(resources[0].uri, 'a')
        self.assertEqual(resources[3].uri, 'd')

+    def test09_resource_list_ordered(self):
+        """Tests for ResourceList with ResourceListOrdered."""
+        i = ResourceList(resources_class=ResourceListOrdered)
+        i.add(Resource('a', timestamp=1))
+        i.add(Resource('d', timestamp=4))
+        i.add(Resource('c', timestamp=3))
+        self.assertEqual(list(i.resources.uris()), ['a', 'd', 'c'])
+        self.assertRaises(ResourceListDupeError, i.add, Resource('a', timestamp=11))
+        self.assertEqual(i.resources['a'].uri, 'a')
+        self.assertEqual(i.resources['a'].timestamp, 1)
+        # With replacement
+        i.add(Resource('a', timestamp=11), replace=True)
+        self.assertEqual(i.resources['a'].uri, 'a')
+        self.assertEqual(i.resources['a'].timestamp, 11)
+
    def test20_as_xml(self):
        rl = ResourceList()
        rl.add(Resource('a', timestamp=1))
@ -181,6 +194,34 @@ class TestResourceList(unittest.TestCase):
        rl = ResourceList()
        self.assertRaises(SitemapParseError, rl.parse, fh=io.StringIO(xml))

-if __name__ == '__main__':
-    suite = unittest.defaultTestLoader.loadTestsFromTestCase(TestResourceList)
-    unittest.TextTestRunner().run(suite)
+    def test33_write(self):
+        # ResourceList
+        rl = ResourceList()
+
+        rl.add(Resource(uri='http://example.com/test/a', timestamp=1))
+        rl.add(Resource(uri='http://example.com/test/b', timestamp=1))
+        rl.add(Resource(uri='http://example.com/test/c', timestamp=1))
+
+        rl_filename = os.path.join(self.tmpdir, 'test33_write_resourcelist.xml')
+        rl.write(basename=rl_filename)
+
+        with open(rl_filename, 'r') as f:
+            s = Sitemap()
+            s.parse_xml(fh=f)
+            self.assertFalse(s.parsed_index)
+
+        # ResourceListIndex
+        rli = ResourceList()
+
+        rli.add(Resource(uri='http://example.com/test/resourcelist00000.xml', timestamp=1))
+        rli.add(Resource(uri='http://example.com/test/resourcelist00001.xml', timestamp=1))
+        rli.add(Resource(uri='http://example.com/test/resourcelist00002.xml', timestamp=1))
+        rli.sitemapindex = True
+
+        rli_filename = os.path.join(self.tmpdir, 'test33_write_resourcelist-index.xml')
+        rli.write(basename=rli_filename)
+
+        with open(rli_filename, 'r') as f:
+            s = Sitemap()
+            s.parse_xml(fh=f)
+            self.assertTrue(s.parsed_index)
--- a/tests/test_resource_list_builder.py
+++ b/tests/test_resource_list_builder.py
@ -1,10 +1,13 @@
 import unittest
 import re
 import os
+from testfixtures import LogCapture
 import time
+
 from resync.resource_list_builder import ResourceListBuilder
+from resync.resource_list import ResourceList
 from resync.resource import Resource
-from resync.mapper import Mapper
+from resync.mapper import Mapper, MapperError


 class TestResourceListBuilder(unittest.TestCase):
@ -138,7 +141,83 @@ class TestResourceListBuilder(unittest.TestCase):
        # Snowman is single char
        self.assertFalse(u'x:snowman_\u2603.txt' in uris)

-if __name__ == '__main__':
-    suite = unittest.defaultTestLoader.loadTestsFromTestCase(
-        TestResourceListBuilder)
-    unittest.TextTestRunner(verbosity=2).run(suite)
+    def test10_add_exclude_patterns(self):
+        """Test add_exclude_patterns method."""
+        rlb = ResourceListBuilder()
+        self.assertEqual(len(rlb.exclude_patterns), 0)
+        rlb.add_exclude_patterns(['aaa', 'bbb'])
+        self.assertIn('aaa', rlb.exclude_patterns)
+        self.assertIn('bbb', rlb.exclude_patterns)
+
+    def test11_compile_excludes(self):
+        """Test _compile_excludes method."""
+        rlb = ResourceListBuilder()
+        self.assertEqual(len(rlb.compiled_exclude_patterns), 0)
+        rlb.exclude_patterns = [r'aaa\d+', r'bbb']
+        rlb._compile_excludes()
+        self.assertEqual(len(rlb.compiled_exclude_patterns), 2)
+        # Error case
+        rlb.exclude_patterns.append('bad regex \\')
+        self.assertRaises(ValueError, rlb._compile_excludes)
+
+    def test12_exclude(self):
+        """Test _exclude method."""
+        rlb = ResourceListBuilder()
+        rlb.add_exclude_patterns(['.*frog.*'])
+        rlb._compile_excludes()
+        self.assertTrue(rlb._exclude('a frog'))
+        self.assertFalse(rlb._exclude('toad'))
+
+    def test13_from_disk_add_path(self):
+        """Test from_disk_add_path method."""
+        # Check sanity check - must have path, resource_list and mapper
+        rlb = ResourceListBuilder(mapper=Mapper())
+        self.assertRaises(ValueError, rlb.from_disk_add_path, path='aaa')
+        self.assertRaises(ValueError, rlb.from_disk_add_path, resource_list=ResourceList())
+        rlb = ResourceListBuilder()
+        self.assertRaises(ValueError, rlb.from_disk_add_path, path='aaa', resource_list=ResourceList())
+        # Check log message
+        rlb = ResourceListBuilder(mapper=Mapper(['http://example.org/', 'tests']))
+        rlb.log_count_increment = 2
+        rl = ResourceList()
+        with LogCapture() as lc:
+            rlb.from_disk_add_path(path='tests/testdata/dir1', resource_list=rl)
+            self.assertIn('from_disk_add_path: 2 files...', lc.records[-1].msg)
+        # text excluding dirs -- just one file under find2 not excluced
+        rlb = ResourceListBuilder(mapper=Mapper(['http://example.org/', 'tests']))
+        rl = ResourceList()
+        rlb.add_exclude_patterns(['find1', 'find3'])
+        rlb._compile_excludes()
+        rlb.from_disk_add_path(path='tests/testdata/find', resource_list=rl)
+        self.assertEqual(len(rl), 1)
+
+    def test14_add_file(self):
+        """Test add_file method."""
+        rlb = ResourceListBuilder(mapper=Mapper(['http://example.org/', 'tests']))
+        rl = ResourceList()
+        rlb.add_exclude_patterns(['.*ro'])
+        with LogCapture() as lc:
+            # escluded
+            rlb.add_file(resource_list=rl, file='frog')
+            self.assertIn("Excluding file 'frog'", lc.records[-1].msg)
+            # mapper error
+            self.assertRaises(MapperError, rlb.add_file, resource_list=rl, file='i-dont-exist')
+            # map OK but doesn't exist
+            rlb.add_file(resource_list=rl, file='tests/i-dont-exist')
+            self.assertIn("Ignoring file 'tests/i-dont-exist'", lc.records[-1].msg)
+            # ignore symlink by default
+            rlb.add_file(resource_list=rl, file='tests/testdata/symlink/dir2/a_file.txt')
+            self.assertIn("Ignoring symlink 'tests/testdata/symlink/dir2/a_file.txt'", lc.records[-1].msg)
+            # ...or not
+            rl = ResourceList()
+            rlb.include_symlinks = True
+            rlb.add_file(resource_list=rl, file='tests/testdata/symlink/dir2/a_file.txt')
+            self.assertEqual(len(rl), 1)
+        # Check hashing
+        rlb = ResourceListBuilder(mapper=Mapper(['http://example.org/', 'tests/testdata/dir1/']),
+                                  set_hashes=['md5', 'sha-1', 'sha-256'])
+        rl = ResourceList()
+        rlb.add_file(resource_list=rl, file='tests/testdata/dir1/file_a')
+        self.assertEqual(rl['http://example.org/file_a'].md5, '6bf26fd66601b528d2e0b47eaa87edfd')
+        self.assertEqual(rl['http://example.org/file_a'].sha1, 'c60a598a5d9e489cf50533eeead6d70f15eafcf8')
+        self.assertEqual(rl['http://example.org/file_a'].sha256, '1c6291bfac0322752c4632ebd69bf6d81d53985fbf5ee54de5cc1fefba6566b6')
--- a/tests/test_resource_list_multifile.py
+++ b/tests/test_resource_list_multifile.py
@ -65,41 +65,37 @@ class TestResourceListMultifile(unittest.TestCase):
        # check the two component sitemaps
        rl1 = ResourceList()
        rl1.read(os.path.join(tempdir, 'sitemap00000.xml'))
-        self.assertEquals(len(rl1), 2)
-        self.assertEquals(rl1.capability, 'resourcelist')
+        self.assertEqual(len(rl1), 2)
+        self.assertEqual(rl1.capability, 'resourcelist')
        self.assertFalse(rl1.sitemapindex)
        i = iter(rl1)
-        self.assertEquals(next(i).uri, 'http://localhost/a')
-        self.assertEquals(next(i).uri, 'http://localhost/b')
+        self.assertEqual(next(i).uri, 'http://localhost/a')
+        self.assertEqual(next(i).uri, 'http://localhost/b')
        rl2 = ResourceList()
        rl2.read(os.path.join(tempdir, 'sitemap00001.xml'))
-        self.assertEquals(len(rl2), 2)
+        self.assertEqual(len(rl2), 2)
        i = iter(rl2)
-        self.assertEquals(next(i).uri, 'http://localhost/c')
-        self.assertEquals(next(i).uri, 'http://localhost/d')
+        self.assertEqual(next(i).uri, 'http://localhost/c')
+        self.assertEqual(next(i).uri, 'http://localhost/d')
        # check the sitemapindex (read just as index)
        rli = ResourceList()
        rli.read(os.path.join(tempdir, 'sitemap.xml'), index_only=True)
-        self.assertEquals(len(rli), 2)
+        self.assertEqual(len(rli), 2)
        i = iter(rli)
-        self.assertEquals(rli.capability, 'resourcelist')
+        self.assertEqual(rli.capability, 'resourcelist')
        self.assertTrue(rli.sitemapindex)
-        self.assertEquals(next(i).uri, 'http://localhost/sitemap00000.xml')
-        self.assertEquals(next(i).uri, 'http://localhost/sitemap00001.xml')
+        self.assertEqual(next(i).uri, 'http://localhost/sitemap00000.xml')
+        self.assertEqual(next(i).uri, 'http://localhost/sitemap00001.xml')
        # check the sitemapindex and components
        rli = ResourceList(mapper=rl.mapper)
        rli.read(os.path.join(tempdir, 'sitemap.xml'))
-        self.assertEquals(len(rli), 4)
-        self.assertEquals(rli.capability, 'resourcelist')
+        self.assertEqual(len(rli), 4)
+        self.assertEqual(rli.capability, 'resourcelist')
        self.assertFalse(rli.sitemapindex)
        i = iter(rli)
-        self.assertEquals(next(i).uri, 'http://localhost/a')
-        self.assertEquals(next(i).uri, 'http://localhost/b')
-        self.assertEquals(next(i).uri, 'http://localhost/c')
-        self.assertEquals(next(i).uri, 'http://localhost/d')
+        self.assertEqual(next(i).uri, 'http://localhost/a')
+        self.assertEqual(next(i).uri, 'http://localhost/b')
+        self.assertEqual(next(i).uri, 'http://localhost/c')
+        self.assertEqual(next(i).uri, 'http://localhost/d')
        # cleanup tempdir
        shutil.rmtree(tempdir)
-
-if __name__ == '__main__':
-    suite = unittest.TestLoader().loadTestsFromTestCase(TestResourceListMultifile)
-    unittest.TextTestRunner(verbosity=2).run(suite)
--- a/tests/test_resource_set.py
+++ b/tests/test_resource_set.py
@ -33,8 +33,3 @@ class TestResourceSet(unittest.TestCase):
        rs.add(Resource('a'))
        self.assertEqual(len(rs), 1)
        self.assertRaises(ResourceSetDupeError, rs.add, Resource('a'))
-
-
-if __name__ == '__main__':
-    suite = unittest.TestLoader().loadTestsFromTestCase(TestResourceSet)
-    unittest.TextTestRunner(verbosity=2).run(suite)
--- a/tests/test_resync-build_script.py
+++ b/tests/test_resync-build_script.py
@ -1,11 +1,6 @@
 import sys
 import unittest
-try:  # python2
-    # Must try this first as io also exists in python2
-    # but in the wrong one!
-    import BytesIO as io
-except ImportError:  # python3
-    import io
+import io

 from resync.resource import Resource
 from resync.resource_list import ResourceList
@ -16,7 +11,7 @@ import subprocess


 def run_resync(args):
-    args.insert(0, 'bin/resync')
+    args.insert(0, './resync-build')
    proc = subprocess.Popen(args, stdout=subprocess.PIPE)
    (out, err) = proc.communicate()
    return(out)
@ -58,7 +53,3 @@ class TestClientLinkOptions(unittest.TestCase):
        self.assertEqual(capl.link('describedby')['href'], 'a')
        self.assertNotEqual(capl.link('up'), None)
        self.assertEqual(capl.link('up')['href'], 'b')
-
-if __name__ == '__main__':
-    suite = unittest.TestLoader().loadTestsFromTestCase(TestClientLinkOptions)
-    unittest.TextTestRunner(verbosity=2).run(suite)
--- a/tests/test_resync-explorer_script.py
+++ b/tests/test_resync-explorer_script.py
@ -10,7 +10,7 @@ except ImportError:  # python3


 def run_resync_explorer(args):
-    args.insert(0, 'bin/resync-explorer')
+    args.insert(0, './resync-explorer')
    proc = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    (out, err) = proc.communicate()
    return(out, err)
@ -21,14 +21,9 @@ class TestClientLinkOptions(unittest.TestCase):
    def test01_help(self):
        """Check that it runs with -h."""
        txt = run_resync_explorer(['-h'])[0]
-        self.assertTrue(txt.startswith(b'Usage: resync-explorer [options] uri'))
+        self.assertRegex(txt, rb'''ResourceSync explorer \(''')

    def test02_error(self):
        """Bad parameter."""
        err = run_resync_explorer([])[1]
-        self.assertRegexpMatches(err, b'FatalError: No source information')
-
-
-if __name__ == '__main__':
-    suite = unittest.TestLoader().loadTestsFromTestCase(TestClientLinkOptions)
-    unittest.TextTestRunner(verbosity=2).run(suite)
+        self.assertRegex(err, rb'''FatalError: No source information''')
--- a/tests/test_sitemap.py
+++ b/tests/test_sitemap.py
@ -1,27 +1,24 @@
+"""Test for resync.sitemap."""
+
+import io
+from testfixtures import LogCapture
 import re
 import sys
 import unittest
-try:  # python2
-    # Must try this first as io also exists in python2
-    # but in the wrong one!
-    import StringIO as io
-except ImportError:  # python3
-    import io
+import xml.etree.ElementTree  # for xml.etree.ElementTree.ParseError
+from defusedxml.ElementTree import parse

 from resync.resource import Resource
 from resync.resource_list import ResourceList
 from resync.sitemap import Sitemap, SitemapIndexError, SitemapParseError

-# etree gives ParseError in 2.7,3.x; ExpatError in 2.6
-etree_error_class = None
-if (sys.version_info < (2, 7)):
-    from xml.parsers.expat import ExpatError
-    etree_error_class = ExpatError
-else:
-    # In python3 this seems only to work with the full class name??
-    # from xml.etree.ElementTree import ParseError
-    import xml.etree.ElementTree
-    etree_error_class = xml.etree.ElementTree.ParseError
+
+class TestSitemapIndexError(unittest.TestCase):
+
+    def test_str(self):
+        """Test str(...) gives just message part."""
+        err = SitemapIndexError("howdy", "this should be the etree")
+        self.assertEqual(str(err), "howdy")


 class TestSitemap(unittest.TestCase):
@ -29,13 +26,33 @@ class TestSitemap(unittest.TestCase):
    def test_01_resource_str(self):
        r1 = Resource('a3')
        r1.lastmod = '2012-01-11T01:02:03Z'
-        self.assertEqual(Sitemap().resource_as_xml(
-            r1), "<url><loc>a3</loc><lastmod>2012-01-11T01:02:03Z</lastmod></url>")
+        self.assertEqual(Sitemap().resource_as_xml(r1),
+                         "<url><loc>a3</loc><lastmod>2012-01-11T01:02:03Z</lastmod></url>")
+        self.assertEqual(Sitemap(spec_version='1.1').resource_as_xml(r1),
+                         "<url><loc>a3</loc><lastmod>2012-01-11T01:02:03Z</lastmod></url>")
+        self.assertEqual(Sitemap(spec_version='1.0').resource_as_xml(r1),
+                         "<url><loc>a3</loc><lastmod>2012-01-11T01:02:03Z</lastmod></url>")
+        self.assertEqual(Sitemap(spec_version='1.1', add_lastmod=True).resource_as_xml(r1),
+                         "<url><loc>a3</loc><lastmod>2012-01-11T01:02:03Z</lastmod></url>")
+        # Now just datetime specified
+        r2 = Resource('aardvark')
+        r2.datetime = '2012-01-11T04:05:06Z'
+        self.assertEqual(Sitemap().resource_as_xml(r2),
+                         '<url><loc>aardvark</loc><rs:md datetime="2012-01-11T04:05:06Z" /></url>')
+        self.assertEqual(Sitemap(spec_version='1.1').resource_as_xml(r2),
+                         '<url><loc>aardvark</loc><rs:md datetime="2012-01-11T04:05:06Z" /></url>')
+        self.assertEqual(Sitemap(spec_version='1.0').resource_as_xml(r2),
+                         '<url><loc>aardvark</loc><lastmod>2012-01-11T04:05:06Z</lastmod></url>')
+        self.assertEqual(Sitemap(spec_version='1.1', add_lastmod=True).resource_as_xml(r2),
+                         '<url><loc>aardvark</loc><lastmod>2012-01-11T04:05:06Z</lastmod><rs:md datetime="2012-01-11T04:05:06Z" /></url>')

    def test_02_resource_str(self):
-        r1 = Resource('3b', 1234.1, 9999, 'ab54de')
-        self.assertEqual(Sitemap().resource_as_xml(
-            r1), "<url><loc>3b</loc><lastmod>1970-01-01T00:20:34.100000Z</lastmod><rs:md hash=\"md5:ab54de\" length=\"9999\" /></url>")
+        r1 = Resource('3b', 1234.1, length=9999, md5='ab54de')
+        self.assertEqual(Sitemap().resource_as_xml(r1),
+                         "<url><loc>3b</loc><lastmod>1970-01-01T00:20:34.100000Z</lastmod><rs:md hash=\"md5:ab54de\" length=\"9999\" /></url>")
+        r1 = Resource('3c', datetime='2013-01-02T13:00:00Z')
+        self.assertEqual(Sitemap().resource_as_xml(r1),
+                         "<url><loc>3c</loc><rs:md datetime=\"2013-01-02T13:00:00Z\" /></url>")

    def test_03_resource_str_hashes(self):
        r1 = Resource('03hashes', 1234.1)
@ -95,7 +112,8 @@ class TestSitemap(unittest.TestCase):
        i = iter(m)
        self.assertEqual(Sitemap().resources_as_xml(i), "<?xml version='1.0' encoding='UTF-8'?>\n<urlset xmlns=\"http://www.sitemaps.org/schemas/sitemap/0.9\" xmlns:rs=\"http://www.openarchives.org/rs/terms/\"><url><loc>a</loc><lastmod>2001-01-01T00:00:00Z</lastmod><rs:md length=\"1234\" /></url><url><loc>b</loc><lastmod>2002-02-02T00:00:00Z</lastmod><rs:md length=\"56789\" /></url></urlset>")

-    def test_10_sitemap(self):
+    def test_10_parse_xml(self):
+        """Test parse_xml method with string XML."""
        xml = '<?xml version=\'1.0\' encoding=\'UTF-8\'?>\n\
 <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/">\
 <url><loc>http://e.com/a</loc><lastmod>2012-03-14T18:37:36Z</lastmod><rs:md hash="md5:Q2hlY2sgSW50ZWdyaXR5IQ==" length=\"12\" /></url>\
@ -110,8 +128,7 @@ class TestSitemap(unittest.TestCase):
            self.assertEqual(r.lastmod, '2012-03-14T18:37:36Z')
            self.assertEqual(r.length, 12)
            self.assertEqual(r.md5, 'Q2hlY2sgSW50ZWdyaXR5IQ==')
-
-    def test_11_parse_2(self):
+        # ..another
        xml = '<?xml version=\'1.0\' encoding=\'UTF-8\'?>\n\
 <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/">\
 <url><loc>/tmp/rs_test/src/file_a</loc><lastmod>2012-03-14T18:37:36Z</lastmod><rs:md length=\"12\" /></url>\
@ -122,6 +139,54 @@ class TestSitemap(unittest.TestCase):
        self.assertFalse(s.parsed_index, 'was a sitemap')
        self.assertEqual(s.resources_created, 2, 'got 2 resources')

+    def test_11_parse_xml_error(self):
+        """Test exceptiona from parse_xml method."""
+        # bad params
+        s = Sitemap()
+        self.assertRaises(ValueError, s.parse_xml)
+        # got a sitemap when told to expect and indexp
+        xml = '<?xml version=\'1.0\' encoding=\'UTF-8\'?>\n\
+<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/">\
+</urlset>'
+        self.assertRaises(SitemapIndexError, s.parse_xml, fh=io.StringIO(xml), sitemapindex=True)
+        # dupe entries DO NOT create an error
+        xml = '<?xml version=\'1.0\' encoding=\'UTF-8\'?>\n\
+<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/">\
+<url><loc>/mouse</loc><lastmod>2020-12-21T00:00:00Z</lastmod><rs:md length=\"12\" /></url>\
+<url><loc>/mouse</loc><lastmod>2020-12-21T00:00:00Z</lastmod><rs:md length=\"12\" /></url>\
+</urlset>'
+        s = Sitemap()
+        i = s.parse_xml(fh=io.StringIO(xml))
+        self.assertEqual(len(i.resources), 2)
+        # preamble rs:md after <url> is error
+        xml = '<?xml version=\'1.0\' encoding=\'UTF-8\'?>\n\
+<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/">\
+<url><loc>/frog</loc><lastmod>2020-12-21T00:01:00Z</lastmod><rs:md length=\"5\" /></url>\
+<rs:md capability=\"resourcelist\"/>\
+<url><loc>/toad</loc><lastmod>2020-12-21T00:02:00Z</lastmod><rs:md length=\"8\" /></url>\
+</urlset>'
+        self.assertRaises(SitemapParseError, s.parse_xml, fh=io.StringIO(xml))
+        # preamble rs:ln after <url> is also error
+        xml = '<?xml version=\'1.0\' encoding=\'UTF-8\'?>\n\
+<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/">\
+<url><loc>/wills</loc><lastmod>2020-12-21T00:01:00Z</lastmod><rs:md length=\"5\" /></url>\
+<rs:ln rel="up" href="http://example.com/resourcesync_description.xml"/>\
+</urlset>'
+        s = Sitemap()
+        self.assertRaises(SitemapParseError, s.parse_xml, fh=io.StringIO(xml))
+        # but random unknown junk should be ignored...
+        xml = '<?xml version=\'1.0\' encoding=\'UTF-8\'?>\n\
+<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/">\
+<junk1>beetle</junk1>\
+<rs:md capability=\"resourcelist\"/>\
+<junk2>fly</junk2>\
+<url><loc>/whale</loc><lastmod>2020-12-21T00:01:00Z</lastmod><rs:md length=\"5\" /></url>\
+<junk3>ant</junk3>\
+</urlset>'
+        s = Sitemap()
+        i = s.parse_xml(fh=io.StringIO(xml))
+        self.assertEqual(len(i.resources), 1)
+
    def test_12_parse_multi_loc(self):
        xml_start = '<?xml version=\'1.0\' encoding=\'UTF-8\'?>\n\
 <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/">\
@ -172,9 +237,9 @@ class TestSitemap(unittest.TestCase):
    def test_15_parse_illformed(self):
        s = Sitemap()
        # ExpatError in python2.6, ParserError in 2.7,3.x
-        self.assertRaises(etree_error_class, s.parse_xml,
+        self.assertRaises(xml.etree.ElementTree.ParseError, s.parse_xml,
                          io.StringIO('not xml'))
-        self.assertRaises(etree_error_class, s.parse_xml,
+        self.assertRaises(xml.etree.ElementTree.ParseError, s.parse_xml,
                          io.StringIO('<urlset><url>something</urlset>'))

    def test_16_parse_valid_xml_but_other(self):
@ -307,6 +372,64 @@ class TestSitemap(unittest.TestCase):
        self.assertEqual(r2.uri, '/tmp/rs_test/src/file_b')
        self.assertEqual(r2.change, None)

-if __name__ == '__main__':
-    suite = unittest.TestLoader().loadTestsFromTestCase(TestSitemap)
-    unittest.TextTestRunner(verbosity=2).run(suite)
+    def test_31_resource_from_etree(self):
+        """Test resource_from_etree method."""
+        # multiple <loc>
+        xml = '''<?xml version=\'1.0\' encoding=\'UTF-8\'?>
+<url xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/">\
+<loc>a_name</loc>
+<loc>another_name_oops</loc>
+</url>'''
+        et = parse(io.StringIO(xml))
+        self.assertRaises(SitemapParseError, Sitemap().resource_from_etree, et, Resource)
+        # no <loc>
+        xml = '''<?xml version=\'1.0\' encoding=\'UTF-8\'?>
+<url xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/">\
+<no_loc_element />
+</url>'''
+        et = parse(io.StringIO(xml))
+        self.assertRaises(SitemapParseError, Sitemap().resource_from_etree, et, Resource)
+        # muktiple <rs:md> not allowed
+        xml = '''<?xml version=\'1.0\' encoding=\'UTF-8\'?>
+<url xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/">\
+<loc>a_name</loc>
+<rs:md type="text/plain" />
+<rs:md something="123" />
+</url>'''
+        et = parse(io.StringIO(xml))
+        self.assertRaises(SitemapParseError, Sitemap().resource_from_etree, et, Resource)
+        # warn is hash invalid
+        with LogCapture() as lc:
+            xml = '''<?xml version=\'1.0\' encoding=\'UTF-8\'?>
+<url xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/">\
+<loc>a_name</loc>
+<rs:md hash="UNKNOWN-TYPE:blah" />
+</url>'''
+            et = parse(io.StringIO(xml))
+            Sitemap().resource_from_etree(et, Resource)
+            self.assertIn('Ignored unsupported hash type UNKNOWN-TYPE', lc.records[-1].msg)
+
+    def test_32_md_from_etree(self):
+        """Test md_from_etree method."""
+        # Warning for unknwon capability
+        with LogCapture() as lc:
+            xml = '''<?xml version=\'1.0\' encoding=\'UTF-8\'?>
+<rs:md xmlns:rs="http://www.openarchives.org/rs/terms/" capability="WHY" />
+'''
+            et = parse(io.StringIO(xml)).getroot()
+            Sitemap().md_from_etree(et)
+            self.assertIn("Unknown capability name 'WHY'", lc.records[-1].msg)
+        # Bad value for change is an error
+        with LogCapture() as lc:
+            xml = '''<?xml version=\'1.0\' encoding=\'UTF-8\'?>
+<rs:md xmlns:rs="http://www.openarchives.org/rs/terms/" change="BAD" />
+'''
+            et = parse(io.StringIO(xml)).getroot()
+            self.assertRaises(SitemapParseError, Sitemap().md_from_etree, et)
+        # length must be an integer
+        with LogCapture() as lc:
+            xml = '''<?xml version=\'1.0\' encoding=\'UTF-8\'?>
+<rs:md xmlns:rs="http://www.openarchives.org/rs/terms/" length="short" />
+'''
+            et = parse(io.StringIO(xml)).getroot()
+            self.assertRaises(SitemapParseError, Sitemap().md_from_etree, et)
--- a/tests/test_source_description.py
+++ b/tests/test_source_description.py
@ -13,7 +13,18 @@ class TestSourceDescription(unittest.TestCase):
        rsd.md_at = None
        self.assertEqual(rsd.as_xml(), '<?xml version=\'1.0\' encoding=\'UTF-8\'?>\n<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/"><rs:ln href="http://example.org/about" rel="describedby" /><rs:md capability="description" /></urlset>')

-    def test02_one_caplist(self):
+    def test02_add(self):
+        rsd = SourceDescription()
+        self.assertEqual(len(rsd), 0)
+        rsd.add(Resource("http://example.org/u1"))
+        rsd.add([Resource("http://example.org/u2"),
+                 Resource("http://example.org/u3")])
+        self.assertEqual(rsd.uris(),
+                         ['http://example.org/u1',
+                          'http://example.org/u2',
+                          'http://example.org/u3'])
+
+    def test03_one_caplist(self):
        rsd = SourceDescription()
        rsd.describedby = "http://example.org/about"
        self.assertEqual(len(rsd), 0)
@ -22,7 +33,7 @@ class TestSourceDescription(unittest.TestCase):
        self.assertEqual(len(rsd), 1)
        self.assertEqual(rsd.as_xml(), '<?xml version=\'1.0\' encoding=\'UTF-8\'?>\n<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/"><rs:ln href="http://example.org/about" rel="describedby" /><rs:md capability="description" /><url><loc>http://example.org/ds1/cl.xml</loc><rs:md capability="capabilitylist" /></url></urlset>')

-    def test03_a_bunch(self):
+    def test04_a_bunch(self):
        rsd = SourceDescription()
        rsd.describedby = "http://example.org/about"
        self.assertEqual(len(rsd), 0)
@ -32,7 +43,7 @@ class TestSourceDescription(unittest.TestCase):
        self.assertEqual(len(rsd), 3)
        self.assertEqual(rsd.as_xml(), '<?xml version=\'1.0\' encoding=\'UTF-8\'?>\n<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/"><rs:ln href="http://example.org/about" rel="describedby" /><rs:md capability="description" /><url><loc>http://example.org/ds1/cl.xml</loc><rs:md capability="capabilitylist" /></url><url><loc>http://example.org/ds2/cl.xml</loc><rs:md capability="capabilitylist" /></url><url><loc>http://example.org/ds3/cl.xml</loc><rs:md capability="capabilitylist" /></url></urlset>')

-    def test04_parse(self):
+    def test05_parse(self):
        xml = '<?xml version=\'1.0\' encoding=\'UTF-8\'?>\n<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:rs="http://www.openarchives.org/rs/terms/"><rs:ln href="http://example.org/about" rel="describedby" /><rs:md capability="description" /><url><loc>http://example.org/ds1/cl.xml</loc><rs:md capability="capabilitylist" /></url><url><loc>http://example.org/ds2/cl.xml</loc><rs:md capability="capabilitylist" /></url><url><loc>http://example.org/ds3/cl.xml</loc><rs:md capability="capabilitylist" /></url></urlset>'
        sd = SourceDescription()
        sd.parse(str_data=xml)
@ -43,8 +54,3 @@ class TestSourceDescription(unittest.TestCase):
        [r1, r2, r3] = sd.resources
        self.assertEqual(r1.uri, 'http://example.org/ds1/cl.xml')
        self.assertEqual(r1.capability, 'capabilitylist')
-
-if __name__ == '__main__':
-    suite = unittest.defaultTestLoader.loadTestsFromTestCase(
-        TestSourceDescription)
-    unittest.TextTestRunner().run(suite)
--- a/tests/test_url_authority.py
+++ b/tests/test_url_authority.py
@ -96,7 +96,3 @@ class TestUrlAuthority(unittest.TestCase):
            'http://a.example.org/sitemap.xml'))
        self.assertFalse(uauth.has_authority_over(
            'http://sub.a.example.org/sitemap.xml'))
-
-if __name__ == '__main__':
-    suite = unittest.TestLoader().loadTestsFromTestCase(TestUrlAuthority)
-    unittest.TextTestRunner(verbosity=2).run(suite)
--- a/tests/test_url_or_file_open.py
+++ b/tests/test_url_or_file_open.py
@ -0,0 +1,37 @@
+"""Tests for resync.url_or_file_open."""
+from .testlib import TestCase, webserver
+import time
+
+from resync.url_or_file_open import NUM_REQUESTS, CONFIG, set_url_or_file_open_config, url_or_file_open
+
+
+class TestUrlOrFileOpen(TestCase):
+
+    def test_set_url_or_file_open_config(self):
+        """Test set_url_or_file_open_config function."""
+        self.assertEqual(CONFIG['bearer_token'], None)
+        self.assertEqual(CONFIG['delay'], None)
+        self.assertNotIn('my_thing', CONFIG)
+        set_url_or_file_open_config('bearer_token', 'open seasame')
+        self.assertEqual(CONFIG['bearer_token'], 'open seasame')
+        set_url_or_file_open_config('my_thing', 'special')
+        self.assertEqual(CONFIG['my_thing'], 'special')
+
+    def test_url_or_file_open(self):
+        """Test basic operation of url_or_file_open function."""
+        # Open file
+        fh = url_or_file_open('tests/testdata/dir1/file_a')
+        self.assertIn(b'I am file a', fh.read())
+        fh.close()
+        with url_or_file_open('file:tests/testdata/dir1/file_b') as fh:
+            self.assertIn(b'I am file b', fh.read())
+        # Open URL
+        with webserver('tests/testdata', 'localhost', 9999):
+            with url_or_file_open('http://localhost:9999/dir2/file_x') as fh:
+                self.assertIn(b'I am the mysterious file_x', fh.read())
+            # test delay of 0.1s
+            set_url_or_file_open_config('delay', 0.1)
+            before = time.time()
+            with url_or_file_open('http://localhost:9999/dir1/file_a') as fh:
+                self.assertIn(b'I am file a', fh.read())
+            self.assertGreater(time.time() - before, 0.099)
--- a/tests/test_w3c_datetime.py
+++ b/tests/test_w3c_datetime.py
@ -126,7 +126,3 @@ class TestW3cDatetime(unittest.TestCase):
                         '2012-03-14T18:47:36Z')
        self.assertEqual(rt('2012-03-14T18:37:36-01:01'),
                         '2012-03-14T17:36:36Z')
-
-if __name__ == '__main__':
-    suite = unittest.TestLoader().loadTestsFromTestCase(TestW3cDatetime)
-    unittest.TextTestRunner(verbosity=2).run(suite)
--- a/tests/testdata/client/dir1/resourcelist.xml
+++ b/tests/testdata/client/dir1/resourcelist.xml
@ -2,16 +2,16 @@
 <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
        xmlns:rs="http://www.openarchives.org/rs/terms/">
  <rs:ln rel="up"
-         href="file:tests/testdata/client/dir1/caps1.xml"/>
+         href="http://localhost:9999/dir1/caps1.xml"/>
  <rs:md capability="resourcelist"
         modified="2013-01-03T15:00:00Z"/>
  <url>
-      <loc>file:tests/testdata/client/dir1/resource1</loc>
+      <loc>http://localhost:9999/dir1/resource1</loc>
  </url>
  <url>
-      <loc>file:tests/testdata/client/dir1/resource2</loc>
+      <loc>http://localhost:9999/dir1/resource2</loc>
  </url>
  <url>
-      <loc>file:tests/testdata/client/dir1/resource3</loc>
+      <loc>http://localhost:9999/dir1/resource3</loc>
  </url>
 </urlset>
--- a/tests/testdata/examples_from_spec_v1_0/archives_ex_2_1.xml
+++ b/tests/testdata/examples_from_spec_v1_0/archives_ex_2_1.xml
--- a/tests/testdata/examples_from_spec_v1_0/archives_ex_2_2.xml
+++ b/tests/testdata/examples_from_spec_v1_0/archives_ex_2_2.xml
--- a/tests/testdata/examples_from_spec_v1_0/archives_ex_3_1.xml
+++ b/tests/testdata/examples_from_spec_v1_0/archives_ex_3_1.xml
--- a/tests/testdata/examples_from_spec_v1_0/archives_ex_3_2.xml
+++ b/tests/testdata/examples_from_spec_v1_0/archives_ex_3_2.xml
--- a/tests/testdata/examples_from_spec_v1_0/archives_ex_4_1.xml
+++ b/tests/testdata/examples_from_spec_v1_0/archives_ex_4_1.xml
--- a/tests/testdata/examples_from_spec_v1_0/archives_ex_5_1.xml
+++ b/tests/testdata/examples_from_spec_v1_0/archives_ex_5_1.xml
--- a/tests/testdata/examples_from_spec_v1_0/archives_ex_6_1.xml
+++ b/tests/testdata/examples_from_spec_v1_0/archives_ex_6_1.xml
--- a/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_1.xml
+++ b/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_1.xml
--- a/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_12.xml
+++ b/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_12.xml
--- a/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_13.xml
+++ b/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_13.xml
--- a/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_14.xml
+++ b/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_14.xml
--- a/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_15.xml
+++ b/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_15.xml
--- a/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_16.xml
+++ b/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_16.xml
--- a/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_17.xml
+++ b/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_17.xml
--- a/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_18.xml
+++ b/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_18.xml
--- a/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_19.xml
+++ b/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_19.xml
--- a/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_2.xml
+++ b/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_2.xml
--- a/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_20.xml
+++ b/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_20.xml
--- a/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_21.xml
+++ b/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_21.xml
--- a/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_22.xml
+++ b/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_22.xml
--- a/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_23.xml
+++ b/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_23.xml
--- a/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_24.xml
+++ b/tests/testdata/examples_from_spec_v1_0/resourcesync_ex_24.xml
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Simeon Warner	3577a202b5	Merge pull request #48 from resync/develop Release 2.0.1	2021-03-23 08:24:21 -04:00
Simeon Warner	6a35c1c95d	Change master->main	2021-03-23 08:21:41 -04:00
Simeon Warner	51b384cc07	Give time to terminate	2021-03-23 08:15:42 -04:00
Simeon Warner	059202695c	Get ready for release 2.0.1	2021-03-23 07:59:32 -04:00
Simeon Warner	e2b0fc9e43	Note about changes	2021-03-22 09:33:33 -04:00
Simeon Warner	4ecdc18703	Merge pull request #46 from resync/issue-45 Use url_or_file_open() in places previously missed	2021-03-18 22:00:01 -04:00
Simeon Warner	9db9950efc	Use url_or_file_open() in places previously missed	2021-03-18 21:51:19 -04:00
Simeon Warner	f5cd751c67	Remove py2.6 cruft	2021-03-18 19:59:50 -04:00
Simeon Warner	49a52478e7	Merge pull request #44 from resync/issue-43 Use --access-token	2021-03-18 19:52:47 -04:00
Simeon Warner	1bb70c597d	Use --access-token	2021-03-18 14:21:44 -04:00
Simeon Warner	8c603fa4cc	Merge pull request #42 from resync/issue-41 Fix handling of missing Content-Length	2021-03-18 14:00:12 -04:00
Simeon Warner	44ea9545dd	Handle missing Content-Length	2021-03-18 13:30:27 -04:00
Simeon Warner	785e851f64	Fix test	2020-12-24 13:27:59 -05:00
Simeon Warner	9ffa84a768	Tidy and test	2020-12-24 13:00:09 -05:00
Simeon Warner	518fd94e5c	Tidy and flesh out tests	2020-12-24 11:07:55 -05:00
Simeon Warner	065363e462	Use OrderedDict now we have python3	2020-12-24 09:00:24 -05:00
Simeon Warner	464f650bea	Add testdata with a symlink	2020-12-22 09:53:19 -05:00
Simeon Warner	a8a193e325	Tidy and tests	2020-12-22 09:50:58 -05:00
Simeon Warner	e0f638d227	Chop out py2 code	2020-12-21 16:27:20 -05:00
Simeon Warner	97f90bc58a	Chop out py2 code	2020-12-21 16:25:22 -05:00
Simeon Warner	42d9269754	Tidy and add tests	2020-12-21 16:22:27 -05:00
Simeon Warner	e3df46b37d	Add tests	2020-12-21 14:55:39 -05:00
Simeon Warner	01d063e280	Bump version number	2020-12-16 18:48:31 -05:00
Simeon Warner	41d16131be	Merge branch 'master' into develop	2020-12-16 18:47:23 -05:00
Simeon Warner	e10416a234	Update instructions	2020-12-16 18:46:59 -05:00
Simeon Warner	df31f42f3f	Merge pull request #40 from resync/develop Release 2.0.0	2020-12-16 18:46:33 -05:00
Simeon Warner	9c1009445a	Update changelog	2020-12-16 17:04:05 -05:00
Simeon Warner	b5c3142435	Use spec_version to correctly handle incrementals using ChangeList	2020-12-16 12:30:04 -05:00
Simeon Warner	9f26daaf74	Tidy examples to match split scripts	2020-12-16 11:19:48 -05:00
Simeon Warner	ce9a15be74	Remove old examples	2020-12-16 11:02:22 -05:00
Simeon Warner	585ef14b01	Merge pull request #39 from resync/v1_1 Add ResourceSync v1.1. support	2020-12-16 08:37:28 -05:00
Simeon Warner	f463e4d5f8	Document add_lastmod	2020-12-15 13:26:42 -05:00
Simeon Warner	fa8db5fec6	Make v1.1 the default, change include_lastmod to add_lastmod	2020-12-15 13:14:30 -05:00
Simeon Warner	b669b3d88f	Change default to v1.1	2020-12-15 11:55:30 -05:00
Simeon Warner	9ddd9d4e1f	Improve comment	2020-12-15 11:46:28 -05:00
Simeon Warner	1ac32a77ea	Chop out python 2.6 and 2.7 hacks	2020-12-15 10:20:58 -05:00
Simeon Warner	5f271767ce	Remove outdated comment	2020-12-13 16:57:30 -05:00
Simeon Warner	1685107039	Change so that Resource.datetime is implemented over ts_datetime timestamp like other timestamps	2020-12-13 16:52:43 -05:00
Simeon Warner	9a789d1b75	Complete working through 1.1 examples	2020-12-13 12:36:35 -05:00
Simeon Warner	3d2a7fcfd9	replace example opener with reader	2020-12-13 12:21:02 -05:00
Simeon Warner	d0dbd4029a	Add 1.1 examples	2020-12-13 11:57:35 -05:00
Simeon Warner	82442f8122	Add spec_version and include_lastmod, add resource.datetime	2020-12-13 11:56:50 -05:00
Simeon Warner	73273f37a8	Move examples from v1.0 into tests/testdata/examples_from_spec_v1_0 and adjust tests	2020-12-13 10:33:30 -05:00
Simeon Warner	9b19782fa0	Fix script names	2020-12-13 10:22:36 -05:00
Simeon Warner	92615b9d93	Change from optparse to argparse, part 3	2020-12-13 10:18:28 -05:00
Simeon Warner	25ba3db42d	Change from optparse to argparse, part 2	2020-12-13 09:31:22 -05:00
Simeon Warner	4d158568b9	Change from optparse to argparse	2020-12-13 09:24:17 -05:00
Simeon Warner	0af0b103dd	Fix comment typo	2020-12-12 12:21:47 -05:00
Simeon Warner	8bef376405	Fix script names	2020-12-12 12:21:28 -05:00
Simeon Warner	50870a65fb	Put icons on same line	2020-12-12 11:25:07 -05:00
Simeon Warner	1a118dd2d2	fix filename	2020-12-12 11:19:25 -05:00
Simeon Warner	8a6ac8d28a	Move scripts back to bin directory	2020-12-12 10:30:29 -05:00
Simeon Warner	8166677045	Move scripts back to bin directory	2020-12-12 10:29:08 -05:00
Simeon Warner	285170eddc	Move version into __init__.py	2020-12-12 10:28:47 -05:00
Simeon Warner	706fb49baa	Update	2020-12-12 10:28:06 -05:00
Simeon Warner	9ece0c5082	Convert to Markdown	2020-12-12 10:27:30 -05:00
Simeon Warner	abc3d7286e	Add --delay option	2020-12-11 11:29:47 -05:00
Simeon Warner	b2c349b943	Update changelog	2020-12-11 08:59:46 -05:00
Simeon Warner	020866bf64	Fix for python 3.8	2020-12-11 08:06:49 -05:00
Simeon Warner	a2db071f3e	Tidy, remove py2 support	2020-12-10 20:19:22 -05:00
Simeon Warner	54ecef29ad	Fix element order in python 3.8	2020-12-10 18:48:59 -05:00
Simeon Warner	82524969ec	Tidy, remove py2 support	2020-12-10 18:09:41 -05:00
Simeon Warner	3391fbef64	Add --access-token, share argument settings	2020-12-10 17:26:36 -05:00
Simeon Warner	3b61c37778	Drop python 2.7	2020-12-10 13:00:32 -05:00
Simeon Warner	0756441d05	Drop py2, sort attributes added to keep output in sort order in Python3.8	2020-12-10 12:57:25 -05:00
Simeon Warner	30ab56b14a	More tidy	2020-12-10 09:16:10 -05:00
Simeon Warner	a15b4c50ce	Fix errors with 3.6, tidy	2020-12-10 07:24:00 -05:00
Simeon Warner	7d2181a31a	Update python versions for testing	2020-12-10 04:47:31 -05:00
Simeon Warner	e1603250bf	Bump version number	2018-10-23 10:59:20 -04:00
Simeon Warner	98292c17b2	Fix date	2018-10-23 10:58:00 -04:00
Simeon Warner	cb92fb4f2a	Tidy	2018-10-23 10:46:41 -04:00
Simeon Warner	e33e7ffbbe	Exclude python2 imports from coverage	2018-10-23 10:46:26 -04:00
Simeon Warner	9de1e5b51e	More tests	2018-10-23 10:45:58 -04:00
Simeon Warner	0125b39757	Merge pull request #35 from UCLALibrary/resource-fetching Enable configuration of timeout and number of retries for resource download attempts	2018-10-02 14:50:15 -04:00
Mark Allen Matney, Jr	a18d221a1f	make pycodestyle happy	2018-10-02 11:44:30 -07:00
Mark Allen Matney, Jr	3d14c31a1c	fix client tests by ensuring that requests.get only sees HTTP URLs	2018-10-02 11:38:52 -07:00
Mark Allen Matney, Jr	956c9eabb2	enable configuration of timeout and number of retries for resource download attempts	2018-10-02 10:48:51 -07:00
Mark Allen Matney, Jr	beabd5b231	remove unused urllib imports	2018-10-01 15:55:57 -07:00
Mark Allen Matney, Jr	01975823dd	fetch resources with 'requests' instead of 'urllib.request'	2018-10-01 13:02:49 -07:00
Simeon Warner	7705041397	Bump version	2017-08-30 14:23:01 -04:00
Simeon Warner	041aead70e	Add explicit --capabilitylist argument	2017-08-30 14:09:29 -04:00
Simeon Warner	d0881b2ec4	Set date	2017-08-30 13:55:39 -04:00
Simeon Warner	e91ca5db08	Remove empty stub	2017-08-30 13:55:17 -04:00
Simeon Warner	8a53a85788	Tidy instructions	2017-08-30 13:44:55 -04:00
Simeon Warner	4eaae3282d	Some more notes	2017-08-30 13:32:30 -04:00
Simeon Warner	cd5e9ce826	Extend pep257 checks	2017-08-30 13:24:51 -04:00
Simeon Warner	f7b78ff73e	No longer support python2.6	2017-08-30 13:20:32 -04:00
Simeon Warner	5868df066a	Tidy to remove __main__ block	2017-08-30 13:11:19 -04:00
Simeon Warner	6e7e81fd48	Use tests.testcase_with_tmpdir to get temd dir for write test	2017-08-30 13:04:23 -04:00
Simeon Warner	84fafbc418	Test truth without explicit is True	2017-08-30 13:04:09 -04:00
Simeon Warner	a04a0aa912	Merge pull request #32 from UCLALibrary/#31 fixes #31	2017-08-30 12:48:44 -04:00
Mark Allen Matney, Jr	8746c3d43b	fixes #31	2017-08-29 22:01:09 -07:00
Simeon Warner	1829858f20	Merge pull request #27 from resync/develop Fix silly error introduced on resync-explorer in v1.0.6	2017-03-27 14:09:35 -04:00