Merge pull request #526 from elbaby/linux-france-gone

change references to the gone linux-france.org domain

.github/FUNDING.yml

@@ -0,0 +1,12 @@
+# These are supported funding model platforms
+
+github: [gilleslamiral] # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
+patreon: # Replace with a single Patreon username
+open_collective: # Replace with a single Open Collective username
+ko_fi: # Replace with a single Ko-fi username
+tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
+community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
+liberapay: # Replace with a single Liberapay username
+issuehunt: # Replace with a single IssueHunt username
+otechie: # Replace with a single Otechie username
+custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']

.github/workflows/docker-image.yml

@@ -0,0 +1,21 @@
+name: Docker Image CI
+
+on:
+  push:
+    branches: [master]
+  pull_request:
+    branches: [master]
+
+jobs:
+  buildx:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v2
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v1
+      - name: Set up Docker Buildx
+        id: buildx
+        uses: docker/setup-buildx-action@v1
+      - name: Available platforms
+        run: echo ${{ steps.buildx.outputs.platforms }}

CREDITS

@@ -1,5 +1,5 @@
 #!/bin/cat
-# $Id: CREDITS,v 1.192 2018/10/04 10:20:47 gilles Exp gilles $
+# $Id: CREDITS,v 1.193 2019/07/31 23:05:10 gilles Exp gilles $
 
 If you want to make a donation to me, imapsync author, Gilles LAMIRAL,
 use any of the following ways:
@@ -9,7 +9,7 @@ b) If you can read french, please use the following wishlist :
   (books will be send with free postal cost)
 
 c) my paypal account is gilles@lamiral.info
-http://imapsync.lamiral.info/S/paypal.shtml
+http://imapsync.lamiral.info/S/donate.shtml
 
 My postal address is:
 Gilles LAMIRAL
@@ -124,7 +124,7 @@ Suggested default --useheader 'Message-ID' --useheader 'Received'
 
 Ameir Abdeldayem
 Gave a patch to sync messages lacking Message-ID
-http://www.linux-france.org/prj/imapsync_list/msg01151.html
+https://web.archive.org/web/20160913193638/http://www.linux-france.org/prj/imapsync_list/msg01151.html
 
 Yousaf Shah
 Contributed by giving the book

FAQ.d/FAQ.APPEND_errors.txt

@@ -1,25 +1,90 @@
 #!/bin/cat
-$Id: FAQ.APPEND_errors.txt,v 1.6 2019/02/16 22:38:49 gilles Exp gilles $
+$Id: FAQ.APPEND_errors.txt,v 1.13 2021/05/29 07:58:44 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
 https://imapsync.lamiral.info/FAQ.d/FAQ.APPEND_errors.txt
 
-=======================================================================
+======================================================================
               Dealing with Imapsync APPEND errors.
-=======================================================================
+======================================================================
 
 Questions answered in this FAQ are:
 
+
+Q. I have an "could not append" error with 
+   "Message contains invalid header" at the end, like:
+   Err 1/20: - msg INBOX/6 {40666} could not append ... NO Message contains invalid header
+   What can I do to transfer these emails?
+
 Q. For some messages, the imapsync log says 
    "could not append", sometimes followed by an explicit message 
    describing what went wrong, or sometimes followed by a not very
    useful message "socket closed while reading data from server"
    What can I do?
 
-R0. Well, the problem is that the "socket closed ..." error message happens
-    in several different issues. So I list here several potential issues
+Q. The append error message is "NO Message contains NUL characters"
+   What can I do?
+
+
+Now the questions again with their answers.
+
+
+======================================================================
+Q. I have an "could not append" error with 
+   "Message contains invalid header" at the end, like:
+   Err 1/20: - msg INBOX/6 {40666} could not append ... NO Message contains invalid header
+   What can I do to transfer these emails?
+
+
+R0. Append in the IMAP protocol is the command to add a message in the folder.
+    The error string "NO. Message contains invalid header" comes from the 
+    destination IMAP server at host2, it doesn't like the message and rejects it.
+   
+R1. Header lines must be in 7bit encoding.
+   If they aren't in 7bit the IMAP server can refuse them as they break 
+   the IMAP RFC specification.
+
+   A solution is to transform all 8bit characters to the Z character with:
+   
+   imapsync ... --regexmess "tr [\x80-\xff] [Z]"
+
+   Caveat: this transformation transforms both the email header and its body.
+   It's not perfect but I guess it's better than no message transferred.
+   So consider using it in a second pass, not the first, as 8-bit 
+   characters in the body message are allowed and changing them when
+   not needed is not a good thing.
+
+
+R2. Some crappy email systems, like virus checkers or any other crappy 
+    software tool dealing with your messages, can add headers to tell 
+    the world that they've done a great job. But sometimes they didn't,
+    and they fucked up your messages by not respecting the standard
+    and added ugly non-rfc compliant headers.
+    
+    To remove an ugly header, let's call it "X-Spam-Report", that spreads over 
+    several lines beginning or, that's the ugly part, not beginning with a space:
+
+    imapsync ... --regexmess 's{X-Spam-Report:.*?\n(^[a-zA-Z0-9\-]+:|^\r?\n)}{$1}xms'
+
+    Thanks to Damien SAUTEREAU for reporting and solving this issue.
+    
+
+======================================================================
+Q. For some messages, the imapsync log says 
+   "could not append", sometimes followed by an explicit message 
+   describing what went wrong, or sometimes followed by a not very
+   useful message "socket closed while reading data from server"
+   What can I do?
+
+R0. Append errors are host2 problems, append is the imap term to copy 
+    a message to the destination account. 
+    A problem is that the associated "socket closed ..." error message happens
+    for several different issues. So I list here several potential issues
     and their solutions if they exist.
+    R1 deals about too long lines in messages on Windows.
+    R2 deals about too long lines in messages on Unix.
+    R3 deals with quota reached.
 
 R1. On Windows, add --regexmess "s,(.{9900}),$1\r\n,g"
 
@@ -77,5 +142,25 @@ OPTIONS
        contains eight-bit characters to quoted-printable encoding.
 
        
-R2. 
-       
+R3. Look for the quota limit on host2. If it is the current mailbox size, don't
+    search elsewhere. Increase the quota on the destination account.
+
+======================================================================
+Q. The append error message is "NO Message contains NUL characters"
+   What can I do?
+
+R. Use:
+
+  imapsync ...  --skipmess "m/(\x00)+\Z/"
+
+The option --skipmess skips messages matching a pattern, the 
+messages are not synced, you just avoid the error messages concerning
+those messages.
+
+If you want to sync them, use instead:
+
+  imapsync ...  --regexmess "s/(\x00)+\Z//g"
+
+
+======================================================================
+======================================================================

FAQ.d/FAQ.Admin_Authentication.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Admin_Authentication.txt,v 1.6 2018/09/09 10:58:16 gilles Exp gilles $
+$Id: FAQ.Admin_Authentication.txt,v 1.13 2022/04/06 10:27:52 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -11,30 +11,45 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Admin_Authentication.txt
 =======================================================================
 
 It can be useful to authenticate without knowing each user password.
+
 Using an admin account to authenticate is not a standard feature
-supported by all imap servers. Sometimes it is implemented via --authuser1
-sometimes not. It depends on the server software and its configuration.
-You have to figure out what is the imap software server,
-find out if it supports an admin account and how, 
-if an admin account is actually configured and
-finally check an authentication via this admin account for
-a standard user.
+supported by all imap servers. Sometimes it is implemented via the 
+--authuser1 option, sometimes not. It depends on the imap software
+server and its configuration.
 
-Known imap server software supporting imap admin authentication:
+You have to follow this road:
+ * Determine what is the imap software server
+ * Find out whether it supports an admin account or not
+ * If no, leave this reading, take a handkerchief and cry
+ * If yes, search how the imap software server supports 
+   an admin account (see below)
+ * Figure out whether an admin account is actually configured or not,
+ * If not then configure it. If you can't configure it: handkerchief.
+ * Finally, check an imapsync authentication via this admin account for
+   a standard user, the option --justlogin is nice for doing this.
+ * Use it for all users.
+ * Next, read their email messages, enter their personal life etc.
+   I'm kidding.
 
-  * Exchange 2003/2007/2010/2013. See the file FAQ.Exchange.txt
-  * Office365. See the file FAQ.Exchange.txt
-  * Gmail. See the file FAQ.XOAUTH2.txt
-  * Dovecot. See the file FAQ.Dovecot.txt
-  * Zimbra. See the file FAQ.Zimbra.txt
-  * UW-imap. See the file FAQ.General.txt
-  * Cyrus-imap. See the file FAQ.General.txt
+Good luck!
+
+Known imap software servers supporting imap admin authentication:
+
+  * Exchange 2003/2007/2010/2013/2016. See the file FAQ.Exchange.txt
+  * Office365.  See the file FAQ.Exchange.txt
+  * Gmail.      See the file FAQ.XOAUTH2.txt
+  * Dovecot.    See the file FAQ.Dovecot.txt
+  * Zimbra.     See the file FAQ.Zimbra.txt
+  * Kerio       See the file FAQ.Kerio.txt
+  * Cyrus-imap. See the file FAQ.Cyrus.txt
+  * James.      See the file FAQ.James.txt
+  * UW-imap.    See the file FAQ.General.txt
   * Sun Java Enterprise System/SunOne/iPlanet. See the file FAQ.General.txt
 
-Known imap server software NOT supporting imap admin authentication:
+Known imap software servers NOT supporting imap admin authentication:
 
-  * Domino Lotus Notes.
-  * SmartCloud Notes.
+  * Domino Lotus Notes. See FAQ.Handkerchief.txt
+  * SmartCloud Notes.   Same joke.
 
 =======================================================================
 =======================================================================

FAQ.d/FAQ.Archiving.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Archiving.txt,v 1.13 2018/12/09 21:26:25 gilles Exp gilles $
+$Id: FAQ.Archiving.txt,v 1.27 2022/05/19 08:56:06 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -11,65 +11,121 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Archiving.txt
 
 Questions answered in this FAQ are:
 
-Q. Can I archive different accounts on the same destination account,
+Q. How can I archive different accounts on the same destination account,
    each account on a separate folder?
 
-Q. How to move emails from one IMAP folder to another either on the 
-  same IMAP server or a different one?
-  For example, all messages older than 1 year, move from INBOX to Archive.
+Q. How can I restore a complete account backuped in a subfolder, like
+   a backup done with the option --subfolder2?
 
-Q. Can imapsync be used to maintain a local offline copy of a 
-   mailbox from host1, eg for backup purposes, so that if the 
-   server fails, then the mailbox could be reinstated,
-   ideally without requiring a second IMAP host?
+Q. How to move emails from one IMAP folder to another, either on the 
+   same IMAP server or on a different one? Example:
+   How to move all messages older than 1 year, from INBOX to Archive?
+
+Q. Can imapsync be used to maintain and restore a local offline copy 
+   of a mailbox, eg for backup purposes, using Mbox or Maildir 
+   format, so that if the server fails, then the mailbox could be 
+   reinstated?
+
+Q. How to archive a whole year in a subfolder?
+
+Q. How to archive a whole year in subfolders but without subsubfolders?
 
 
+Now the questions again with their answers.
+
+https://imapsync.lamiral.info/FAQ.d/FAQ.Archiving.txt
 =======================================================================
-Q. Can I archive different accounts on the same destination account,
+Q. How can I archive different accounts on the same destination account,
    each account on a separate folder?
 
-R. Yes. Use --subfolder2 
+R. Use --subfolder2 
 
  --subfolder2   str  : Syncs the whole host1 folders hierarchy under the
                        host2 given folder str.
                        (It does it internally by adding two --regextrans2 
                        options before all others.)
 
-Example:
+Examples:
 
-  imapsync ... --user1 foo --subfolder2 foo
+To backup the account Foo in a subfolder called FooArchiv:
 
-  imapsync ... --user1 bar --subfolder2 bar
+  imapsync ... --user1 Foo --subfolder2 FooArchiv
+
+To backup the account Bar in a subfolder called BarArchiv:
+
+  imapsync ... --user1 Bar --subfolder2 BarArchiv
+
+In case you need a very strict sync, add options 
+--delete2 --delete2foldersonly "FooArchiv"
+(or "BarArchiv" for the second example).
+It will delete on account2 what is not on account1 but only in the right place, 
+ie, the subfolder "FooArchiv".
+
+Examples:
+
+To mirror the account Foo in a subfolder called FooArchiv:
+
+  imapsync ... --user1 Foo --subfolder2 FooArchiv --delete2 --delete2foldersonly "FooArchiv" 
+
+To mirror the account Bar in a subfolder called BarArchiv:
+
+  imapsync ... --user1 Bar --subfolder2 BarArchiv --delete2 --delete2foldersonly "BarArchiv" 
+
+See how to restore those with the next Q/R.
 
-In case you need a strict sync, add --delete2 --delete2foldersonly "foo"
-(or "bar"), il will delete on account2 what is not on account1 but only
-in the right place, ie, the subfolder "foo".
-  
-  imapsync ... --user1 foo --subfolder2 foo --delete2 --delete2foldersonly "foo" 
-  
 =======================================================================
-Q. How to move emails from one IMAP folder to another either on the 
-  same IMAP server or a different one?
-  For example, move all messages older than 1 year from INBOX to Archive.
+Q. How can I restore a complete account backuped in a subfolder, like
+   a backup done with the option --subfolder2?
+
+R. Use --subfolder1
+
+ --subfolder1   str  : Syncs the host1 folders hierarchy under folder str
+                        to the root hierarchy of host2.
+                        It's the counterpart of a sync done by --subfolder2
+                        when doing it in the reverse order. 
+                        Backup/Restore scenario:
+                        Use --subfolder2 str for a backup to the folder str
+                        on host2. Then use --subfolder1 str for restoring 
+                        from the folder str, after inverting 
+                        host1/host2 user1/user2 values.
+
+Examples:
+
+To restore the whole folder FooArchiv hierarchy into the Foo account:
+
+  imapsync ... --user2 Foo --subfolder1 FooArchiv
+
+To restore the whole folder BarArchiv hierarchy into the Bar account:
+
+  imapsync ... --user2 Bar --subfolder1 BarArchiv
+
+Read the previous FAQ item "How can I archive different accounts on the same 
+destination account, each account on a separate folder?"
+
+
+=======================================================================
+Q. How to move emails from one IMAP folder to another, either on the 
+   same IMAP server or on a different one? Example:
+   How to move all messages older than 1 year, from INBOX to Archive?
  
 R1. Solution:
 
-  imapsync ... --minage 365 --folder Inbox --f1f2 INBOX=Archive --delete1 --noexpungeaftereach
+  imapsync ... --minage 365 --folder INBOX --f1f2 INBOX=Archive --delete1 --noexpungeaftereach
 
-Add option --noexpungeaftereach if speed is a concern.
-Add option --delete1 if you want to move messages, instead of just copy/sync them.
+Use the option --noexpungeaftereach if speed is a concern.
+Use the option --delete1 if you want to move messages, instead of just copy/sync them.
 
 R2. See also file FAQ.Folders_Mapping.txt
 
 
 https://imapsync.lamiral.info/FAQ.d/FAQ.Archiving.txt
 =======================================================================
-Q. Can imapsync be used to maintain a local offline copy of a 
-   mailbox from host1, eg for backup purposes, so that if the 
-   server fails, then the mailbox could be reinstated,
-   ideally without requiring a second IMAP host?
+Q. Can imapsync be used to maintain and restore a local offline copy 
+   of a mailbox, eg for backup purposes, using Mbox or Maildir 
+   format, so that if the server fails, then the mailbox could be 
+   reinstated?
 
-R1. No. Imapsync plays with imap servers only.
+R1. No. Imapsync plays with IMAP servers only.
 
 R2. Imapsync can't but rick-sanders-imap-tools can!
     See https://github.com/andrewnimmo/rick-sanders-imap-tools
@@ -85,6 +141,40 @@ R2. Imapsync can't but rick-sanders-imap-tools can!
     is more dangerous than doing no backups at all. No backups makes
     people act in a safer way, well, usually...
     
-=======================================================================
-=======================================================================
+R3. Use isync or mbsync
+    https://isync.sourceforge.io/
 
+R4. Use ImportExportTools for Mozilla Thunderbird by Paolo Kaosmos. 
+    ImportExportTools does not do IMAP as a destination.
+
+R5. There is a free online service (French language) allowing you to do
+    a Maildir or EML or HTML backup at 
+    https://www.powermail.fr/sauvegarder-une-boite-mail/
+    Prefer Maildir or EML format if you want to be able to restore the 
+    archive. HTML format will be ok to only view the messages, not to
+    restore them to an imap server account.
+
+I have never used any of this tools so asking me for support with them 
+won't go far away.
+
+=======================================================================
+Q. How to backup a whole year in a subfolder?
+
+R. To backup the whole year 2018 in a subfolder called ARCHIVE_2018:
+
+  imapsync ... --search "SENTSINCE 1-Jan-2018 SENTBEFORE 31-Dec-2018" \
+      --subfolder2 ARCHIVE_2018 
+
+=======================================================================
+Q. How to archive a whole year in subfolders but without subsubfolders?
+
+R. Instead of the solution with --subfolder2, use:
+
+  imapsync ... --search "SENTSINCE 1-Jan-2018 SENTBEFORE 31-Dec-2018" \
+             --sep2 _ --regextrans2 's{(.*)}{Archive_$1_2018}' 
+
+For each folder "FFFF" in the source account, it will create a destination
+folder called "Archive_FFFF_2018", at the top level hierarchy.
+
+=======================================================================
+=======================================================================

FAQ.d/FAQ.Authentication_failure.txt

@@ -1,9 +1,9 @@
 #!/bin/cat
-$Id: FAQ.Authentication_failure.txt,v 1.9 2019/01/11 13:13:08 gilles Exp gilles $
+$Id: FAQ.Authentication_failure.txt,v 1.16 2022/08/17 09:18:17 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
-https://imapsync.lamiral.info/FAQ.d/FAQ.Admin_Authentication.txt
+https://imapsync.lamiral.info/FAQ.d/FAQ.Authentication_failure.txt
 
 =======================================================================
               Imapsync authentication issues
@@ -17,21 +17,25 @@ Host1 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2
 Q. Imapsync fails with the following error, what can I do?
    Host2 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2 BAD Invalid characters in atom 
 
+Now the questions again with their answers.
+
 
 =======================================================================
 Q. Imapsync fails with the following error, what can I do?
 Host1 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2 NO [AUTHENTICATIONFAILED] Authentication failed
 
 
-R. One over four imapsync syncs ends up quickly with the error message 
-   "Authentication failed" or "NO LOGIN failed" or similar. 
+R. One over four imapsync synchronizations end up quickly with the error message 
+   "Authentication failed" or "NO LOGIN failed" or a similar message.
    Authentication failure is the primary failure with imapsync
    and since nothing tangible can be done without authentication, 
    this stage must succeed to go further.
 
-Here are some advices to get you pass this difficult stage of authentication:
+Here is some advice to get you to pass this difficult stage of authentication:
 
-  * Triple check each credential parameter, there are three parameters at each side:
+  * Triple-check each credential parameter. It's not always a password problem,
+    it can be the wrong host or the wrong user.
+    There are three parameters on each side:
      * triple check --host1
      * triple check --user1
      * triple check --password1
@@ -39,7 +43,7 @@ Here are some advices to get you pass this difficult stage of authentication:
      * triple check --user2
      * triple check --password2
 
-   * If you can authenticate successfully with an other imap client software 
+   * If you can authenticate successfully with another imap client software 
      like Thunderbird or Outlook or Sparrow then it is a very good sign to 
      authenticate successfully with imapsync. Examine the parameters of 
      this other imap client and copy them as is for imapsync.
@@ -48,41 +52,76 @@ Here are some advices to get you pass this difficult stage of authentication:
      At the beginning of the output, imapsync dumps all its command
      line parameters; it's the line after "Command line used:".
      With --showpasswords, imapsync prints the passwords received 
-     instead of the string MASKED. It helps for debugging quoting issues.
+     instead of the string MASKED. It helps with debugging quoting issues.
      Option --showpasswords shows passwords again when the IMAP dialog
      is dumped by --debugimap option. Search for a line like 
-     "Sending: 2 LOGIN test1 secret1" (secret1 is the password here)
-
+     "Sending: 2 LOGIN test1 secret1" (secret1 is the password here).
 
    * It is sometimes very hard to quote correctly unusual characters,
-     especially on Windows. See 
+     especially on Windows (and Mac sometimes). See 
      https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Windows.txt
      https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt
-     The quickest trick may be to change the password temporally
-     with easy characters like the classical alphabet, a long
-     string still ensure strong security.
+     https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Mac.txt
+     
+     The quickest trick may be to change the password temporarily
+     with easy characters like the classical alphabet, one long
+     string still ensures strong security.
 
    * Sometimes some servers announce they support LOGIN but it
-     actually fails because the authentication mechanism
+     fails because the authentication mechanism
      working is something else like CRAM-MD5 or PLAIN. 
      So:
        * Try --authmech1 CRAM-MD5 (or --authmech2 CRAM-MD5)
        * Try --authmech1 PLAIN    (or --authmech2 PLAIN)
 
-   * If you want to play manually the IMAP protocol on your server, 
-     here is an example with the command telnet, user test1
-     and password secret1 on the host test.lamiral.info.
+    * Manually test a login via ssl.
+      Use either ncat or telnet-ssl or openssl commands, like in the 
+      following examples with imap.gmail.com server:
 
-     telnet test.lamiral.info 143
+  ncat --ssl -C imap.gmail.com 993
+  telnet-ssl -z ssl  imap.gmail.com 993
+  openssl s_client -crlf  -connect imap.gmail.com:993
+
+Replace imap.gmail.com by your imap server.
+The three commands do the same work here.
+
+Typical dialog for an imap LOGIN command:
+  
+* OK Gimap ready for requests from 78.196.254.58 q1mb175739668wix
+a LOGIN "gilles.lamiral@gmail.com" "secret" 
+* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE ... ESEARCH
+a OK gilles.lamiral@gmail.com Gilles Lamiral authenticated (Success)
+b LOGOUT
+* BYE LOGOUT Requested
+b OK 73 good day (Success)
+
+The client part you have to type is 
+a LOGIN ... 
+b LOGOUT
+while replacing ... by your credentials values,
+other lines are the server responses. Letters 
+a and b are identifiers of IMAP commands, you have to 
+type them but you can chose any string instead of a or b.
+
+Some other examples:
+
+     ncat --ssl test.lamiral.info 993
+     c1 LOGIN test1 "secret1"
+     c2 LOGOUT
+
+     openssl s_client -crlf  -connect test.lamiral.info:993
      c1 LOGIN test1 "secret1" 
      c2 LOGOUT
 
-     Same example using an ssl conneection:
-     
+     ncat --ssl -C outlook.office365.com 993
+     a LOGIN gilles.lamiral@outlook.com topsecret
+     b LOGOUT
+
      telnet-ssl -z ssl test.lamiral.info 993
      c1 LOGIN test1 "secret1" 
      c2 LOGOUT
-   
+
+
 =======================================================================
 Q. Imapsync fails with the following error, what can I do?
    Host2 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2 BAD Invalid characters in atom 

FAQ.d/FAQ.Bandwidth.txt

@@ -0,0 +1,48 @@
+
+$Id: FAQ.Bandwidth.txt,v 1.3 2021/05/20 11:46:07 gilles Exp gilles $
+
+This documentation is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Bandwidth.txt
+
+
+=======================================================================
+             Imapsync bandwidth used
+=======================================================================
+
+
+Questions answered in this FAQ are:
+
+Q. What is the bandwidth used by imapsync?
+
+Now the questions again with their answers.
+
+=======================================================================
+Q. What is the bandwidth used by imapsync?
+
+R. From the host where imapsync runs, imapsync opens two imap
+connections, one with the source account at host1, one with the
+destination account at host2.
+   
+So, the global bandwidth used by an imapsync transfer is twice the
+volume of the source account, one volume to download the messages from
+host1, one volume to upload those messages to host2.
+
+If the host2 is already filled with the messages, imapsync doesn't
+transfer them and then the volume transferred is small, this volume is
+just made up of the IMAP commands needed to identify the messages on
+both sides.
+   
+There is no local cache of the email messages, except when a message
+is very big; it is then temporarily saved locally between the download
+and the upload.
+   
+The biggest message seen so far on the online service I call /X is
+3.08 GiB while the biggest message transferred is 1.51 GiB. So I
+suspect a bug here. Drop me a note if you encounter the same issue,
+I'll then dig into it, ie, I'll create a 2 or 3 GiB big message and
+play with it :-)
+
+
+=======================================================================
+=======================================================================

FAQ.d/FAQ.Big_Mailbox.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Big_Mailbox.txt,v 1.2 2018/07/25 10:37:30 gilles Exp gilles $
+$Id: FAQ.Big_Mailbox.txt,v 1.8 2022/04/23 13:45:48 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -10,24 +10,47 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Big_Mailbox.txt
             Imapsync tips to deal with huge mailboxes
 =====================================================================
 
-
 Questions answered in this FAQ are:
 
 Q. How to deal with huge mailboxes, whose size is over dozens of GB?
 
-
-
 Now the questions again with their answers.
 
 =====================================================================
 Q. How to deal with huge mailboxes, whose size is over dozens of GB?
 
-R. It should be ok with imapsync. In case imapsync seems to stall
-   when sizing the folders or before syncing a huge folder and 
-   you wander if it is doing something or just frozen, you can
-   add option --debugimap. Option  --debugimap will show what is 
-   currently done, it's quite a big output but it helps waiting, 
-   saying to ourself
-   "ok it's long but it's working, let's wait a little more".
-   The real purpose of --debugimap is to show genuine IMAP 
-   commands used and their responses.
+R1. It should be ok with imapsync.  
+
+Sometimes with big folders, imapsync seems to stall when sizing the
+folders or maybe before syncing a huge folder. So you wonder whether
+imapsync is doing something or being just frozen. A solution to
+increase the verbosity is to add the option --debugimap. Option
+--debugimap shows what is currently done, it's quite a big output but
+it helps to wait, saying to ourselves "ok it's long but it's working,
+let's wait a little more".  The real purpose of --debugimap is to show
+genuine IMAP commands used and their responses. The problem with
+--debugimap is that it is very verbose and so with a big mailbox it
+generates a very BIG logfile.
+
+R2. If it is not ok with imapsync, the sync finally fails, then 
+consider splitting the whole sync into smaller chunks using 
+the option --search
+
+For example, select messages only for a given year:
+
+...
+..imapsync ... --search "SENTSINCE 1-Jan-2017 SENTBEFORE 31-Dec-2017"
+  imapsync ... --search "SENTSINCE 1-Jan-2018 SENTBEFORE 31-Dec-2018"
+  imapsync ... --search "SENTSINCE 1-Jan-2019 SENTBEFORE 31-Dec-2019"
+
+R3. Use option --useuid
+Option --useuid speeds up the process but --useuid implies --usecache
+and --usecache may trigger some issues sometimes. Read:
+https://imapsync.lamiral.info/FAQ.d/FAQ.Use_cache.txt
+
+Don't use --useuid in Docker context, the cache will be destroyed 
+each time, or put it in a stable place, I don't know how to do it,
+I just know it's possible and not very difficult.
+
+=====================================================================
+=====================================================================

FAQ.d/FAQ.Capacity_Planning.txt

@@ -0,0 +1,103 @@
+$Id: FAQ.Capacity_Planning.txt,v 1.6 2022/04/23 13:45:59 gilles Exp gilles $
+
+This documentation is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Capacity_Planning.txt
+
+
+======================================================================
+            Imapsync tips for Capacity Planning.
+======================================================================
+
+
+I plan to go to a distributed architecture for the online service. I'm
+doing some capacity planning. Imapsync takes memory, cpu and bandwidth
+in a relatively deterministic values.
+
+My current question is: Shall I use 
+* N   2GB hosts
+* N/2 4GB hosts
+* N/4 8GB hosts
+
+Let's do some observations and maths
+
+The observations are done on the standalone imapsync online which 
+characteristics are:
+
+CPU: Intel i5-2300 with 4 cores
+
+RAM: 16 GB
+
+NET: 100 Mbps symmetrical, 12.5 MBytes/s symmetrical, 
+     so 25 MBytes/s max for a global imapsync rate.
+
+Disks: I don't know.
+
+System: FreeBSD 11.4
+
+===== CPU =====
+
+The CPU can be an issue. On average, an imapsync run takes 5% of the
+overall cpu time for a Intel i5-2300 with 4 cores.  It implies 20
+imapsync runs is ok on the current online host before the cpus become
+the bottleneck.  As a rule of thumb, imapsync takes 20% of a cpu core.
+
+In the Intel i5-2300 4 cores, so far, the maximum number of imapsync 
+processes has been 68, which is 3 times what the system should 
+support in a standard imapsync workload. For this workload, the 
+imapsync performamces were not good, the server could not handle the load
+and was even out of order for a while. For 40 imapsync processes, the
+the performances are ok.
+
+===== RAM =====
+
+The RAM can be an issue. On average, an imapsync run takes 250 MB.  So
+4 imapsync processes per GB is the limit before swapping to disk,
+which is a known phenomenon telling when memory becomes the bottleneck.
+16 GB allows 64 imapsync processes.
+
+On the 16GB, so far, the maximum memory usage taken by imapsync processes was
+13 GB. For that value, the one minute load was 10 or more, the number of 
+imapsync processes was around 40, the total bandwidth was around 19 MiB/s.
+
+===== LINK =====
+
+The Bandwidth I/O can be an issue.  The "Average bandwidth rate" value
+given by imapsync at the end of a transfer, and also the bandwidth
+rate given during the sync on the ETA line, is the total size of all
+messages copied divided by the time passed. If imapsync is run between
+two foreign imap servers then the total size transferred on the
+network link is twice this value, one time when getting the message
+from host1, rx, and one time sending the message to host2, tx.
+
+On average, an imapsync runs at 3 Mbps both ways, rx and tx, so 6 Mbps
+in total.  So a 100 Mbps symmetric link allows 33 imapsync processes
+before the link becomes the bottleneck.
+
+The best minutes observed so far are a global 187 Mpbs rate (~23 MiB/s) 
+on a 100 Mbps symmetrical link done by 28 imapsync processes in parallel, 
+6.7 Mbps per process at that minute.  But the best minutes observed for a
+single imapsync process is at 103 Mbps (~13 MiB/s), the rx/tx link limit.
+
+===== DISKS =====
+
+The harddisk I/O can be an issue. I don't measure it yet because
+imapsync doesn't perform heavy I/0 where it runs. Well, I don't know,
+no measure is no knowledge, just guesses.
+
+
+===== Repartition ======
+
+
+Let's consider we have a bunch of different hosts able to run imapsync 
+processes. How should we distribute imapsync jobs among them?
+
+A simple rule is "do not add more load to a host when one of the resources
+has reached its maximal. The resources are memory, bandwidth, cpu, disk.
+I ignore the disk part for now.
+
+maximum number of imapsync processes for a host 
+= min( 4 * RAM_in_GB, 10 * nb_cores, bandwidth_in_mbps / 3 )
+
+
+

FAQ.d/FAQ.Contacts_Calendars.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Contacts_Calendars.txt,v 1.11 2018/09/26 22:25:32 gilles Exp gilles $
+$Id: FAQ.Contacts_Calendars.txt,v 1.17 2022/05/31 22:21:55 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -11,43 +11,29 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Contacts_Calendars.txt
 
 Questions answered in this FAQ are:
 
-Q. Can I copy or sync Calendars or Contacts with imapsync?
-Q. Can I copy or sync Chat messages with imapsync?
-Q. How can I avoid copying Calendar or Contacts folders?
+Q. Can I copy or sync Contacts, Calendars, Tasks or Chat messages with imapsync?
 Q. How can I copy or synchronize Calendars or Contacts?
+Q. How can I avoid copying Calendar or Contacts folders?
 
 Now the questions again with their answers.
 
 =======================================================================
-Q. Can I copy or sync Calendars or Contacts with imapsync?
+Q. Can I copy or sync Contacts, Calendars, Tasks or Chat messages with imapsync?
 
-R. No, Imapsync can't migrate Contacts and Calendars. 
-  It's because most IMAP servers don't get contacts and calendars 
+R. No, Imapsync can't migrate Contacts, Calendars, Tasks nor Chat messages.
+  It's because most IMAP servers don't get Contacts and Calendar 
   events via IMAP. In other words, messages synced by imapsync from 
-  Calendars or Contacts folders are not used by email servers to set 
-  or get the contacts or calendars. 
-  No way via IMAP, no way via imapsync. 
-  So it's a good idea to avoid syncing contacts and calendars.
-  But see the question below
+  Contacts/Calendars/Tasks/Chat folders are not used by email servers to set 
+  or get the contacts, calendars, tasks, or chat messages. 
+  No way via IMAP, no way via imapsync.
+  
+  So it's a good idea to avoid syncing contacts and calendars when
+  using imapsync. See at the end of this document
+  "How can I avoid copying Calendar or Contacts folders?"
+  
+  But see also the question below
   "How can I copy or synchronize Calendars or Contacts?".
 
-=======================================================================
-Q. Can I copy or sync Chat messages with imapsync?
-
-R. No, Imapsync can't migrate Chat messages (those in GSuite for example)
-   The explanation is the same as with Calendars or Contacts.
-
-=======================================================================
-Q. How can I avoid copying Calendar or Contacts folders?
-
-R1. You can avoid synchronizing Calendar or Contacts folders with
-  the --exclude option. First you have to search what is their
-  exact name. The folders listing printed by imapsync at the
-  beginning of a run will surely help to find their names. 
-  Assuming their names are "Calendars" and "Contacts" use then:
-
-    imapsync ... --exclude "Calendar" --exclude "Contacts" 
-
 =======================================================================
 Q. How can I copy or synchronize Calendars or Contacts?
 
@@ -66,17 +52,55 @@ b) From Gmail to Gmail, export and import calendars in ical format,
    Contacts can be copied using a csv file. See the help page
    http://support.google.com/mail/bin/topic.py?hl=en&topic=1669027
 
-c) Etc. Search the web. There's also specific tools and paid services.
+c) For Outlook, use caldavsynchronizer.
+   Homepage: http://caldavsynchronizer.org/
+   Outlook CalDav Synchronizer is a free Outlook Plugin, which synchronizes 
+   events, tasks, and contacts between Outlook and Google, SOGo, Nextcloud or 
+   any other CalDAV or CardDAV server. 
+   Supported Outlook versions are Office 365, 2019, 2016, 2013, 2010, and 2007.
+   Free and Open Source software tool.
+
+
+d) Etc. Search the web. There are also specific tools and paid services.
    There's no silver bullet to migrate Calendars and Contacts,
    if you find one, tell me!
-   Also consider asking experts at Sumatra company 
+   Also, consider asking experts at the Sumatra company 
    https://twitter.com/sumatra_dev
    http://sumatra.com/
    
-d) I plan to make tools for that but so far nothing has began
+e) I plan to make tools for that but so far nothing has begun
    more than this discussion:
-http://www.linux-france.org/prj/imapsync_list/msg01797.html
-http://www.linux-france.org/prj/imapsync_list/msg01811.html
+http://linux-france.tk/prj/imapsync_list/msg01797.html
+http://linux-france.tk/prj/imapsync_list/msg01811.html
+
+=======================================================================
+Q. How can I avoid copying Calendar or Contacts folders?
+
+R. You can avoid synchronizing Calendar or Contacts folders with
+  the --exclude option. First, you have to search what is their
+  exact name. The listing of folders printed by imapsync at the
+  beginning of a run will surely help to find their names. 
+  Assuming their names are "Calendars" and "Contacts", use:
+
+    imapsync ... --exclude "Calendar" --exclude "Contacts" 
+
+  Be aware that this syntax will exclude all folders that have
+  Calendar or Contacts in their names, as a substring,
+  like "MyCalendarZ" or "TheContactsImDoneWith". 
+  Also, case matters: "calendar" does not match "Calendar".
+  
+  To strictly avoid only "Calendar" or "Contacts" use 
+  the following
+
+  On windows:
+  
+    imapsync ... --exclude "^Calendar$" --exclude "^Contacts$" 
+
+  On Unix:
+  
+    imapsync ... --exclude '^Calendar$' --exclude '^Contacts$'
+    or
+    imapsync ... --exclude "^Calendar\$" --exclude "^Contacts\$" 
 
 
 =======================================================================

FAQ.d/FAQ.Cyrus.txt

@@ -0,0 +1,127 @@
+#!/bin/cat
+$Id: FAQ.Cyrus.txt,v 1.2 2021/02/19 13:39:03 gilles Exp gilles $
+
+=======================================================================
+             Imapsync tips for the Cyrus imap server
+=======================================================================
+
+This documentation is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Cyrus.txt
+
+
+Questions answered in this FAQ are:
+
+Q. How to migrate from cyrus with an admin account?
+
+Q. I'm migrating from WU to Cyrus, and the mail folders are under
+   /home/user/mail but the tool copies everything in /home/user, how
+   can i avoid that?
+
+Q. I'm migrating from WU to Cyrus, and the mail folders are under
+   /home/user/mail directory.  When imapsync creates the folders in
+   the new cyrus imap server, it makes a folder "mail" and below that
+   folder puts all the mail folders the user have in /home/user/mail,
+   i would like to have all those folders directly under INBOX.
+
+Q. Migrating from Groupwise to Cyrus
+
+Now the questions again with their answers.
+
+
+======================================================================
+Q. How to migrate from cyrus with an admin account?
+
+R. Use:
+ 
+ imapsync ... \
+   --authuser1 admin_user  ----password1 admin_user_password \
+   --user1 foo_user --ssl1
+
+Instead of --ssl1 the alternative --tls1 can be used.
+With --authuser1, the option --authmech1 PLAIN is set
+automatically, you don't have to add it.
+
+PLAIN authentication is the only way to go with --authuser1 for now.
+So don't use --authmech1 SOMETHING with --authuser1 admin_user,
+it will not work.
+Same behavior with the --authuser2 option.
+
+Do not forget the option --ssl1 or --tls1 since PLAIN auth is only
+supported with ssl encryption most of the time. But it can
+work without --ssl1 nor --tls1 if PLAIN is permitted in clear text
+transmissions (the normal mode).
+
+Add the AdminAccount to admins line in /etc/imapd.conf
+Give AdminAccount lrswipkxtecda to the Cyrus Imap account 
+being migrated from, "joe" here.
+
+
+Here is an example:
+ imapsync \
+	--host1 server1 \
+	--user1 joe \
+	--authuser1 AdminAccount \
+	--password1 AdminAccountPassword \
+	--ssl1 \
+	--host2 server2 \
+	--user2 joe \
+	--password2 joespassonserver2 \
+	--exclude "^user\."
+
+=======================================================================
+Q. I'm migrating from WU to Cyrus, and the mail folders are under
+   /home/user/mail but the tool copies everything in /home/user, how
+   can i avoid that?
+
+Two solutions:
+
+R. Use
+   imapsync ... --include '^mail'
+
+R. or (better)
+   imapsync ... --subscribed --subscribe
+
+=======================================================================
+Q. I'm migrating from WU to Cyrus, and the mail folders are under
+   /home/user/mail directory.  When imapsync creates the folders in
+   the new cyrus imap server, it makes a folder "mail" and below that
+   folder puts all the mail folders the user have in /home/user/mail,
+   i would like to have all those folders directly under INBOX.
+
+R. Use 
+   imapsync ... --regextrans2 's/^mail/INBOX/' --dry
+   look at the simulation and if all transformations seem
+   good then remove the --dry option.
+
+
+=======================================================================
+Q. Migrating from Groupwise to Cyrus
+
+R. By Jamie Neil:
+
+I eventually managed to get the mail to migrate without errors using the 
+following options:
+
+--maxage 3650
+  - some messages just don't seem to want to transfer and produce the 
+    perl errors I mentioned before. This prevents the errors, but the
+    bad messages don't transfer.
+
+Even though the mail migrated OK, there are a couple of gotchas with 
+Groupwise IMAP:
+
+1) Some of the GW folders are not real folders and are not available
+to IMAP, the main problem one being "Sent Items". I could find no way
+of coping the contents of these folders. The nearest I got was to
+create a "real" folder and copy/move the sent items into it, but
+imapsync still didn't see the messages (I think because there is
+something funny about the reported dates/sizes).
+
+It think this problem has been rectified in GW6.5.
+
+2) Version 6.0.1 of the Groupwise Internet Connector sucks. I was 
+getting server aborts when I pushed it a bit hard! I eventually had to
+upgrade to 6.0.4 which seems to be a lot more stable.
+
+=======================================================================

FAQ.d/FAQ.DBmail.txt

@@ -0,0 +1,22 @@
+#!/bin/cat
+$Id: FAQ.DBmail.txt,v 1.1 2020/11/04 08:43:05 gilles Exp gilles $
+
+This document is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.DBmail.txt
+
+=======================================================================
+              Imapsync tips for DBmail accounts.
+=======================================================================
+
+
+=======================================================================
+Q. How to migrate from or to DBmail?
+
+R. In order to avoid duplicates, use:
+
+  imapsync ...  --useheader Message-Id
+
+More explanations and details at
+https://imapsync.lamiral.info/FAQ.d/FAQ.Duplicates.txt
+

FAQ.d/FAQ.Dates.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Dates.txt,v 1.9 2018/11/04 09:27:17 gilles Exp gilles $
+$Id: FAQ.Dates.txt,v 1.11 2019/12/23 12:46:50 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -26,7 +26,11 @@ Q. We have found that the time and date displayed have been changed to
 R. This is the case by default with some email readers like:
    - Outlook 2003
    - Ipad
-
+   - Iphone
+   - spark https://sparkmailapp.com/  (iphone app)
+   - Edison mail  https://apps.apple.com/us/app/email-edison-mail/id922793622 (iphone app)
+   - ...
+   
 but not with:
 
    - Mutt
@@ -34,15 +38,17 @@ but not with:
    - Zimbra
    - Gmail
 
-A first thing to keep in mind. 
-Imapsync does not touch any byte of messages unless told to 
-do so by special options like --regexmess or --pipemess.
-Messages on both parts should be exactly identical because
-imapsync does a verbatim sync of all messages.
+First, a thing to keep in mind. 
+
+Imapsync does not touch any byte of messages unless told to do so by
+special and rarely used options like --regexmess or --pipemess.
+Messages on both parts should be exactly identical because imapsync 
+does a verbatim sync of all messages, by default.
 
 Now I explain the whole picture about dates of messages.
 
 There are several different dates attached to any message.
+
 First, there is the "Date:" header, which belongs to the message itself.
 You can see this "Date:" header line by asking "View->Message Source"
 in most email client software (Ctrl-u in Thunderbird).
@@ -52,37 +58,40 @@ MUA can be Outlook, Thunderbird, a webmail or Mutt etc.
 The "Date:" header is usually the date the message was written 
 or sent the first time, at the writer side. 
 This "Date:" header line is never changed by any transfer or copy.
-If an email reader uses the "Date:" header for displaying the date of a 
-message then no problem should arise.
 
-Another date is the internal date. In IMAP, the internal date is handled
-by the imap server and normally it corresponds to the arrival date in the mailbox.
-This date is not part of the email message itself, it's a metadata kept
-by the imap server.
-The IMAP protocol allows the internal date to be set by a email client
+If an email reader uses the "Date:" header for displaying the date of
+a message then no problem should arise.
+
+Another date is the "internal date". In IMAP, the internal date is
+handled by the imap server and normally it corresponds to the arrival
+date in the mailbox.  This date is not part of the email message
+itself, it is a metadata kept by the imap server.
+
+The IMAP protocol allows the internal date to be set by an email client
 when a message is delivered by the imap APPEND command.
 Imapsync synchronizes these internal dates, by default, so internal dates 
 on host2 should then be the same as the internal dates on host1.
-If an email reader uses the internal date for displaying the date of a 
-message then the sync date problem only occurs when the host2 server software
-ignores the internal date given by imapsync during the APPEND imap 
-command. It happens with some imap servers.
 
-There are also the "Received:" header lines. Each time a message travels
-a SMTP server, this one adds a "Received:" header line. Sometimes 
-some email clients use the last "Received:" header date as the date of the
-message. And some IMAP servers software add a "Received:" line after
-and imap transfer. If those both conditions are met then the date
-displayed become the transfer date even if imapsync 
+If an email reader uses the internal date for displaying the date of a
+message then the sync date problem only occurs when the host2 imap
+server ignores the internal date given by imapsync during the APPEND
+imap command. It happens with some imap servers. Bad luck.
+
+There are also the "Received:" header lines. Each time a message
+travels a SMTP server, this one adds a "Received:" header
+line. Sometimes some email clients use the last "Received:" header
+date as the date of the message. And some IMAP servers software add a
+"Received:" line after and imap transfer. If those both conditions are
+met then the date displayed become the transfer date even if imapsync
 has done its best to keep all the dates synchronized. Bad luck.
 
 
 Solutions: 
 a) Use another email client or configure it in order that it sorts messages
-by sent dates, the "Date:" header line.
+   by sent dates, the "Date:" header line.
 
 b) Use a imap server that respects the imap RFC and accepts 
-the internal date set by imapsync.
+   the internal date set by imapsync.
 
 c) Try to understand why the email client software shows another date 
    than the "Date:" header line.

FAQ.d/FAQ.David_Tobit.txt

@@ -0,0 +1,54 @@
+#!/bin/cat
+$Id: FAQ.David_Tobit.txt,v 1.5 2022/04/05 13:48:52 gilles Exp gilles $
+
+This documentation is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.David_Tobit.txt
+
+
+=======================================================================
+             Imapsync tips for David Tobit imap server
+=======================================================================
+
+
+
+=======================================================================
+Q. Migrating to David Tobit V12
+   (Banner: "* OK IMAP4rev1 David.fx Mail Access Server MA-12.00a (0271)")
+
+  imapsync ... --sep2 / --create_folder_old
+
+Thanks to Ludwig Behm
+
+=======================================================================
+Q. Migrating from David Tobit V10 (DvISE Mail Access Server MA-...)
+
+R. Use the following options:
+
+  imapsync ... --prefix1 ""    --sep1 /  --idatefromheader ^
+               --nofoldersizes --useuid --nocheckmessageexists
+
+=======================================================================
+Q. Migrating from David Tobit V8
+   (Banner: "* OK IMAP4rev1 DvISE Mail Access Server MA-8.10a (0126)")
+
+First try above V10 solution since improvments have been made
+to support Tobit.
+
+R. Use the following options :
+
+  imapsync ... --prefix1 INBOX. --sep1 / --subscribe --subscribed 
+
+=======================================================================
+Q. Migrating from Tobit David Server 6 
+   (Banner: "* OK IMAP4rev1 DvISE Mail Access Server MA-6.60a (0118)")
+
+First try above V10 solution since improvments have been made
+to support Tobit.
+
+R. Look at the discussion:
+http://linux-france.tk/prj/imapsync_list/msg00583.html
+http://linux-france.tk/prj/imapsync_list/threads.html#00583
+patch saved in ./patches/imapsync-1.337_tobit_V6.patch
+
+=======================================================================

FAQ.d/FAQ.Docker.txt

@@ -0,0 +1,160 @@
+#!/bin/cat
+# $Id: FAQ.Docker.txt,v 1.10 2021/07/12 21:22:14 gilles Exp gilles $
+
+This document is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Docker.txt
+
+======================================================================
+           Installing and using imapsync docker image
+======================================================================
+
+Questions answered in this FAQ are:
+
+
+Q. How can I install and use the imapsync Docker image on my amd64 system?
+
+Q. How can I install and use the imapsync Docker image on my i386 system?
+
+Q. How can I install or update only the imapsync Docker image on my system?
+
+Q. Where is the imapsync Docker hub location?
+
+Q. How can I mask the passwords on the command line without mounting
+   inside the container?
+
+Q. Why imapsync isn't logging to a file in Docker context. 
+   How can I allow it?
+
+Q. Any tips for the Docker Mailcow distribution?
+
+Q. How can I build my own image?
+
+Q. What do you do to update the docker hub image?
+
+Now the questions again with their answers.
+
+======================================================================
+Q. How can I install and use the imapsync Docker image on my amd64 system?
+
+R. Install Docker on your system. Once Docker is installed on your system,
+   all you have to do in order install and run imapsync is the command line:
+
+  docker run gilleslamiral/imapsync imapsync <usual imapsync arguments>
+
+======================================================================
+Q. How can I install and use the imapsync Docker image on my i386 system?
+
+R1. You can't do it directly for now. It will throw this error when running it:
+
+  docker run gilleslamiral/imapsync imapsync
+  standard_init_linux.go:211: exec user process caused "exec format error"
+
+I plan to make the image multi-plateform. Drop me a note if you are
+eager to have it.
+
+R2. Build the image on a i386 system with the exact same Dockerfile provided
+and run it on your i386 system.
+
+======================================================================
+Q. How can I install or update only the imapsync Docker image on my system?
+
+R. To install or update the imapsync image, run:
+
+  docker pull gilleslamiral/imapsync
+
+======================================================================
+Q. Where is the imapsync Docker hub location?
+
+R. Here:
+
+  https://hub.docker.com/r/gilleslamiral/imapsync/
+
+======================================================================
+Q. How can I mask the passwords on the command line without mounting 
+   anything inside the container?
+
+R. Use a file defining the environment variables 
+   IMAPSYNC_PASSWORD1 and IMAPSYNC_PASSWORD2
+
+  cat ./secret.txt
+IMAPSYNC_PASSWORD1=secret1
+IMAPSYNC_PASSWORD2=secret2
+
+Use that file like this for a run:
+
+  docker run --env-file ./secret.txt gilleslamiral/imapsync imapsync ...
+
+See:
+https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file
+https://stackoverflow.com/a/30494145/491175
+
+======================================================================
+Q. Why imapsync isn't logging to a file in Docker context. 
+   How can I allow it?
+
+R. In Docker context, writing the log is disabled by default because you 
+   don't have an easy access to the logfile after the sync. The logfile is 
+   inside the docker and no longer available after the sync, unless
+   a special mount is done before.
+
+Starting with imapsync release 2.113 the logging can be turned on 
+by using the option --log. You can add access to the logfile with a mount 
+or similar. See https://docs.docker.com/storage/volumes/"
+
+See also
+https://github.com/imapsync/imapsync/issues/283
+
+======================================================================
+Q. Any tips for the Docker Mailcow distribution?
+
+R. With the Mailcow distribution, imapsync is running inside a Docker 
+   container. Quoting Mathilde:
+
+To make it work with Mailcow, options should be added like this:
+
+  --regexflag=s/\\Indexed//gi
+
+Without spaces, without quotes.
+
+See the original Mathilde's comment:
+
+https://github.com/imapsync/imapsync/issues/201#issuecomment-559500077
+
+======================================================================
+Q. How can I build my own image?
+
+R. Use the following command to build the image:
+
+   docker build -t gilleslamiral/imapsync . 
+ 
+   with the Dockerfile in the current directory.
+
+The latest Dockerfile can be found at
+https://imapsync.lamiral.info/INSTALL.d/Dockerfile
+or 
+https://hub.docker.com/r/gilleslamiral/imapsync
+or 
+https://imapsync.lamiral.info/INSTALL.d/INSTALL.Docker_build.txt
+
+
+======================================================================
+Q. What do you do to update the docker hub image?
+
+R. I do in my imapsync directory:
+
+    make docker_build
+    make docker_upload_docker_hub
+    
+Then I go to
+https://hub.docker.com/r/gilleslamiral/imapsync
+
+I edit and update the docker "Readme" with the last Dockerfile I used.
+I also update INSTALL.Docker_build.txt with the last Dockerfile
+
+I also update the last Dockerfile on the imapsync site with:
+
+   make upload_index
+
+======================================================================
+======================================================================

FAQ.d/FAQ.Domino.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-# $Id: FAQ.Domino.txt,v 1.10 2018/05/24 11:34:30 gilles Exp gilles $
+# $Id: FAQ.Domino.txt,v 1.13 2020/09/06 20:33:46 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -10,9 +10,23 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Domino.txt
                Imapsync tips for Domino.
 =======================================================================
 
+Questions answered in this FAQ are:
+
+Q. How to use imapsync from Domino Notes to XXX?
+
+Q. How to use imapsync from XXX to Domino Notes?
+
+Q. Whan I use --maxage imapsync sees no messages at all.
+   What can I do?
+
+Q. My usual email software client like Thunderbird sees more messages
+   than imapsync. How can it be possible?
+
+Now the questions again with their answers:
+
 
 =======================================================================
-Q. From Domino Notes to XXX
+Q. How to use imapsync from Domino Notes to XXX?
 
 R1. Use --domino1, like this:
 
@@ -30,7 +44,7 @@ On Unix use:
   imapsync ... --sep1 '\' --prefix1 '' --messageidnodomain
 
 =======================================================================
-Q. From XXX to Domino Notes
+Q. How to use imapsync from XXX to Domino Notes
 
 R1. Use --domino2, like this:
 
@@ -91,4 +105,38 @@ For Domino 853FP6 on Linux, we used this command on Unix:
 
 
 =======================================================================
+Q. When I use --maxage option, imapsync sees no messages at all.
+   What can I do?
+
+R. Use --noabletosearch
+
+  imapsync ... --noabletosearch
+
+Same thing with --minage
+
+=======================================================================
+Q. On Domino this errors occurs: 
+   "Login failure, cannot convert Mailbox: code 02:46"
+
+R. Follow
+   https://atnotes.de/index.php?topic=44888.0
+
+Use:
+
+   load convert -e mail\mailfilename.nsf
+
+=======================================================================
+Q. My usual email software client like Thunderbird sees more messages
+   than imapsync. How can it be possible?
+
+R. (Issue given and solved by Falko Veith)
+   "As you might know, it is possible to replicate databases, aka mailboxes,
+   across domino servers. THAT is the explanation. Thunderbird and some Notes 
+   clients may be connected to one server and imapsync to another.
+   As funny and simply the solution is, as mysterious is why those two 
+   databases are not completely in sync - new emails are replicated but not
+   the old ones. One mystery solved - and a new one to go.
+
+Thanks to Falko Veith for this question and the answer!
+
 =======================================================================

FAQ.d/FAQ.Dovecot.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Dovecot.txt,v 1.6 2018/05/24 11:34:30 gilles Exp gilles $
+$Id: FAQ.Dovecot.txt,v 1.14 2022/06/16 09:06:34 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -11,6 +11,43 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Dovecot.txt
 =======================================================================
 
 
+Questions answered in this FAQ are:
+
+Q. Is there imapsync specific options to deal with Dovecot?
+
+Q. The login fails with the error:
+   Err 1/1: Host2 failure: Error login on [...] with user [...] auth [LOGIN]: timeout waiting 120s for data from server (5x)
+   What can it be?
+
+Q. Subfolders are not created by imapsync and the error is
+   "Mailbox doesn't allow inferior mailboxes"
+
+Q. How to migrate to Dovecot with an admin/MasterUser account?
+
+Q. I have a mailbox that "dovecot backup" gives errors on that
+   I can't get rid of: "UID inserted in the middle of mailbox".
+   What can I do?
+
+Q. Imapsync encounters timeouts and Dovecot log says
+   "mailpiler dovecot: imap(xxx): Error: Timeout (180s) while waiting for dotlock for transaction log file ..."
+   What can I do?
+
+Now the questions again with their answers.
+
+=======================================================================
+Q. Is there imapsync specific options to deal with Dovecot?
+
+R. Dovecot is ok with imapsync, nothing special is needed, just the
+   basic options, ie, the credentials, are needed.
+
+=======================================================================
+Q. The login fails with the error:
+   Err 1/1: Host2 failure: Error login on [...] with user [...] auth [LOGIN]: timeout waiting 120s for data from server (5x)
+   What can it be?
+
+R. It can be special characters in the password. Change it with only
+   normal characters, characters from a-z A-Z 0-9
+
 =======================================================================
 Q. How to migrate to Dovecot with an admin/MasterUser account?
 
@@ -18,8 +55,49 @@ R. Dovecot uses the same syntax as uw-imap
 
   imapsync ... --user2="loginuser*admin_user" --password2 "admin_user_password"
 
-To setup a Dovecot MasterUser see 
-http://wiki2.dovecot.org/Authentication/MasterUsers
+To setup a Dovecot MasterUser follow the document
+https://doc.dovecot.org/configuration_manual/authentication/master_users/
+
+=======================================================================
+Q. I have a mailbox that "dovecot backup" gives errors on that
+   I can't get rid of: "UID inserted in the middle of mailbox".
+   What can I do?
+
+R. Solution, run:
+
+  doveadm -v deduplicate -m -u martin mailbox A/Posted
+
+it will get rid of the duplicates (it can also be run without "-m")
+and this also allows "dovecot backup" to work.
+
+Thanks to Martin Ward for this input!
+
+=======================================================================
+Q. Subfolders are not created by imapsync and the error is
+   "Mailbox doesn't allow inferior mailboxes"
+
+R. Dovecot usually supports subfolders with messages and subfolders,
+   it's a Dovecot configuration issue.
+
+If you can change the Dovecot configuration dovecot.conf, 
+then try this:
+
+mail_location = maildir:~/Maildir:LAYOUT=fs
+
+See details at:
+https://superuser.com/questions/813362/sub-folders-in-dovecot
+https://wiki2.dovecot.org/MailLocation/Maildir
+
+
+=======================================================================
+Q. Imapsync encounters timeouts and Dovecot log says
+   "mailpiler dovecot: imap(xxx): Error: Timeout (180s) while waiting for dotlock for transaction log file ..."
+   What can I do?
+
+R. It looks like an NFS issue with Dovecot. 
+   Solve it with Dovecot support or use something else than NFS.
+   I'll be glad to share here any solution you find.
+   (Thanks to Mate Skoblar for reporting this issue)
 
 =======================================================================
 =======================================================================

FAQ.d/FAQ.Duplicates.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Duplicates.txt,v 1.18 2019/02/14 17:22:10 gilles Exp gilles $
+$Id: FAQ.Duplicates.txt,v 1.25 2021/10/06 20:21:13 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -9,12 +9,57 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Duplicates.txt
             Imapsync tips about duplicated messages issues.
 =======================================================================
 
+
+Questions answered in this FAQ are:
+
+Q. Without imapsync, I made several copies that partially failed and it
+   ended with many duplicates/triplicates messages or more. Can I clean
+   up the account with imapsync and how?
+
+Q. How does imapsync identify messages and duplicates?
+
+Q. How can I know if imapsync will generate duplicates on a second run?
+
+Q: I found multiple copies, duplicates, when I run imapsync twice or
+   more. What the hell is happening?
+   
+Q. imapsync calculates 479 messages in a folder but only transfers 400 
+   messages. What is happening?
+
+Q. imapsync doesn't synchronize duplicates by default but I want to.
+   How can I synchronize duplicates?
+
+Q. How can I remove duplicates in a lone account?
+
+
+Now the questions again with their answers.
+
+=======================================================================
+Q. Without imapsync, I made several copies that partially failed and it
+   ended with many duplicates/triplicates messages or more. Can I clean
+   up the account with imapsync and how?
+
+R. Yes. 
+   See the Q/R "How can I remove duplicates in a lone account?" below.
+
+=======================================================================
+Q. How does imapsync identify messages and duplicates?
+
+R. Imapsync by default identify messages by their headers "Message-Id" 
+   and "Received". Usually, for a given message, "Message-Id" appears one
+   time while multiple "Received" headers are common.
+   
+   For imapsync, messages with the same "Message-Id" and "Received" headers 
+   are consider identical, ie, duplicates.
+
+
 =======================================================================
 Q. How can I know if imapsync will generate duplicates on a second run?
 
 R. To see if imapsync will generate duplicates on a second run, start
-   a second run with --dry option added. imapsync will then show if it 
-   would mistakenly copy messages again, but without really copying them
+   a second run with --dry option added. With --dry, imapsync will 
+   show whether it would mistakenly copy messages again, but without 
+   really copying them:
 
      imapsync ... --dry 
 
@@ -31,11 +76,12 @@ Messages transferred              : 0
 Messages skipped                  : 1555
 
 =======================================================================
-Q: Multiple copies, duplicates, when I run imapsync twice ore more. 
+Q: I found multiple copies, duplicates, when I run imapsync twice or
+   more. What the hell is happening?
 
 R0. First, some explanations to understand the issue. 
 Normally and by default, imapsync doesn't generate duplicates.
-So if it does  generate duplicates it means a problem occurs
+So, if it does  generate duplicates it means a problem occurs
 with message identification. It happens sometimes with IMAP 
 servers changing the "Message-Id" header line or one or more
 of the "Received:" header lines in the header part of messages. 
@@ -62,12 +108,12 @@ of all messages already transferred.
 
 If you end with: 
 Messages skipped                  : 0
-don't go on, it means imapsync is still suffering to
+then don't go on, it means imapsync is still suffering to
 identify messages.
 
 If you end with many messages skipped then it's very
-good and now you can safely resync the mailboxe
-and get rid of the dupplicates messages on host2 with:
+good and now you can safely resync the mailbox
+and get rid of the duplicate messages on host2 with:
 
     imapsync ... --useheader "Message-Id" --delete2duplicates
 
@@ -80,7 +126,7 @@ to identify and compare messages in folders.
 Instead of some headers,  --useuid tell imapsync to use 
 the imap UIDs given by imap servers on both sides.
 To avoid duplicates on next runs, imapsync uses a local cache
-where it keeps UIDs already transfered. 
+where it keeps UIDs already transferred. 
 
   imapsync ... --useuid
 
@@ -95,11 +141,12 @@ A solution? Two solutions.
 The easiest is --delete2 if you are permitted to use it. 
 Option --delete2 removes messages on host2
 that are not on host1. So, with --delete2 you go for resyncing all 
-messages again but all previously transferred messages are deleted,
-but also messages previously there without imapsync. 
-So --useuid --delete2 is easy to remove duplicates but not for 
-all contexts. The host2 account must be considered as a strict
-replication of the host1 account, ie, not active.
+messages again. All previously transferred messages are deleted,
+but also messages previously there without imapsync.
+So --useuid --delete2 is an easy way to remove duplicates but it
+is not suitable in all contexts. The good context is that the host2 
+account must be considered as a strict replication of the host1 
+account, ie, host2 not active yet.
 
 A second solution, better if R3 works (see R3 below), is to build
 the cache before using --useuid
@@ -170,7 +217,7 @@ trick to keep the --useheader filtering behavior is to use
 
 =======================================================================
 Q. imapsync calculates 479 messages in a folder but only transfers 400 
-   messages. What's happen?
+   messages. What is happening?
 
 R1. Unless --useuid is used, imapsync considers a header part 
    of a message to identify a message on both sides.
@@ -188,11 +235,45 @@ R2. With option --useuid imapsync doesn't use headers to identify
     messages on both sides but it uses their imap uid identifier.
     In that case duplicates on host1 are also transferred on host2.
 
+=======================================================================
+Q. imapsync doesn't synchronize duplicates by default but I want to.
+   How can I synchronize duplicates?
+
+R1. Use the option  --syncduplicates
+
+R2. Use the option --useuid
+   If you have already synchronized two mailboxes without --useuid then
+   using it right away will generate duplicates on host2. To avoid that
+   behavior, you have to perform a first run with --usecache to build
+   the local UID cache. Then the next runs with --useuid 
+
+   There are potentially issues with --usecache. They can be solved.
+   Read the document FAQ.Use_cache.txt 
+   https://imapsync.lamiral.info/FAQ.d/FAQ.Use_cache.txt
+
+   So to finalise how to synchronize duplicates:
+   
+   imapsync ... --tmpdir . --usecache 
+
+   imapsync ... --tmpdir . --useuid
+   imapsync ... --tmpdir . --useuid
+   ...
+
+   Here --tmpdir value is the dot "." meaning "current directory".
+   Surrounding it with double-quotes is optional.
+
+   If the two mailboxes haven't been already synchronized then the
+   first run with --usecache is useless.
+
 =======================================================================
 Q. How can I remove duplicates in a lone account?
 
-R. Just run imapsync on the same account with option --delete2duplicates,
-   ie, with host1 == host2, user1 == user2, password1 == password2
+R. In order to remove duplicates in a lone account, just run imapsync 
+   on the same account as source and destination, plus the 
+   option --delete2duplicates, ie, with 
+   host1 == host2, user1 == user2, password1 == password2
+ 
+  imapsync ... --delete2duplicates
 
 =======================================================================
 =======================================================================

FAQ.d/FAQ.Emptying.txt

@@ -1,38 +1,153 @@
 #!/bin/cat
-$Id: FAQ.Emptying.txt,v 1.6 2018/05/24 11:34:30 gilles Exp gilles $
+$Id: FAQ.Emptying.txt,v 1.14 2022/06/08 11:08:39 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
-https://imapsync.lamiral.info/FAQ.d/
+
+https://imapsync.lamiral.info/FAQ.d/FAQ.Emptying.txt
 
 =======================================================================
-               Imapsync tip to empty an account.
+               Imapsync tips about deletions.
 =======================================================================
 
+Questions answered in this FAQ are:
+
+Q. How to delete all emails of all folders of an account with imapsync?
+
+Q. How to delete all folders of an account with imapsync?
+
+Q. How to delete some folders of an account with imapsync?
+
+Q. How to revert a sync that went to the wrong account?
+
+Q. How to delete emails older than 90 days with imapsync?
+
+Q. How to delete emails matching a certain criterium with imapsync?
+
+Q. Is there a way to delete the destination folders when the source 
+   folders are no longer there?
+
+Now the questions again with their answers.
+
 =======================================================================
 Q. How to delete all emails of all folders of an account with imapsync?
 
-R. Use the same account on both sides with option --delete
+R. Use the same account on both sides with option --delete1
    --noexpungeaftereach
 
-Option --noexpungeaftereach is just to speed up the deletion.
+Option --noexpungeaftereach is just to speed up the deletions.
 
 Example:
 
-  ./imapsync \
-    --host1 test.lamiral.info --user1 test --password1 secret \
-    --host2 test.lamiral.info --user2 test --password2 secret \
-    --delete --noexpungeaftereach
+  imapsync \
+    --host1 test.lamiral.info --user1 foo --password1 secret \
+    --host2 test.lamiral.info --user2 foo --password2 secret \
+    --delete1 --noexpungeaftereach
 
 =======================================================================
 Q. How to delete all folders of an account with imapsync?
 
-R. Use an already empty account as source and sync this account
-   to the account you want to clear, with --delete2folders 
+R. Use the same account on both sides with the options 
+    --delete1 --noexpungeaftereach --delete1emptyfolders
+    
+  imapsync ... --delete1 --noexpungeaftereach --delete1emptyfolders
+
+It won't delete the folder INBOX since INBOX is mandatory in imap.
+
+
+=======================================================================
+Q. How to delete some folders of an account with imapsync?
+
+R. Use the same account on both sides with the options 
+    --delete1 --noexpungeaftereach --delete1emptyfolders
+    like when deleting all folders BUT select only the folders
+    you want to delete by using options to select folders,
+    like --folder or --folderrec or --include or --exclude 
+
+Example, to delete the folder Blabla
+
+  imapsync ... --delete1 --noexpungeaftereach --delete1emptyfolders --folder Blabla
+
+To delete the four folders Kalender Kontakte Aufgaben Notizen:
+
+  imapsync ... --include 'Kalender|Kontakte|Aufgaben|Notizen' \
+                 --delete1 --noexpungeaftereach --delete1emptyfolders
+
+I strongly recommend to use --dry first!
+
+To get an overview of how to select folders, see
+https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Selection.txt
+
+=======================================================================
+Q. How to revert a sync that went to the wrong account?
+
+R. If all the wrong emails have some common characteristic that is not
+ shared by the good one, then you can search for them and delete them
+ using the same account as source and destination. For example, if all
+ the messages have the same email destination address as
+ "john_smith@example.com", then you can remove them with the following
+ command:
+   
+   imapsync ... --search "TO john_smith@example.com" --delete1 
+
+ Be very careful with deletions! Since this section deals with something
+ that was made wrong, you're in good place to make a new one!
+ 
+ There is many possibilities to select messages with different criterium.
+ See some of them by reading the document 
+ https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt
+
+ If the wrongly emails messages synced have no special common 
+ characteristic then take an handkerchief and cry.
+
+=======================================================================
+Q. How to delete emails older than 90 days with imapsync?
+
+R. Use the same account on both sides with option --delete1
+   --noexpungeaftereach --minage 90 
 
 Example:
 
-  ./imapsync ... --delete2folders --foldersizes
+  imapsync \
+    --host1 test.lamiral.info --user1 foo --password1 secret \
+    --host2 test.lamiral.info --user2 foo --password2 secret \
+    --delete1 --noexpungeaftereach --minage 90 
 
 =======================================================================
+Q. How to delete emails matching a certain criterium with imapsync?
+
+R. Use the --search option to select the messages you want to delete 
+   and then use the same account on both sides with the suplementary 
+   options --delete1 --noexpungeaftereach
+   To select the messages read the document FAQ.Messages_Selection.txt
+   at https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt
+
+
+Example:
+
+  imapsync \
+    --host1 test.lamiral.info --user1 foo --password1 secret \
+    --host2 test.lamiral.info --user2 foo --password2 secret \
+    --delete1 --noexpungeaftereach --search "SENTBEFORE 31-Dec-2010"
+
+======================================================================
+Q. Is there a way to delete the destination folders when the source 
+   folders are no longer there?
+
+R. Yes, use --delete2folders
+
+ --delete2folders    : Delete folders in host2 that are not in host1.
+                       For safety, first try it like this, it is safe:
+                       --delete2folders --dry --justfolders --nofoldersizes
+                       and see what folders will be deleted.
+
+ --delete2foldersonly   reg : Delete only folders matching the regex reg.
+                              Example: --delete2foldersonly "/^Junk$|^INBOX.Junk$/"
+                              This option activates --delete2folders
+
+ --delete2foldersbutnot reg : Do not delete folders matching the regex rex.
+                              Example: --delete2foldersbutnot "/Tasks$|Contacts$|Foo$/"
+                              This option activates --delete2folders
+
 =======================================================================
+=======================================================================

FAQ.d/FAQ.Exchange.txt

@@ -1,98 +1,89 @@
 
-$Id: FAQ.Exchange.txt,v 1.49 2018/12/03 23:09:28 gilles Exp gilles $
+$Id: FAQ.Exchange.txt,v 1.80 2022/04/05 13:48:52 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
 https://imapsync.lamiral.info/FAQ.d/FAQ.Exchange.txt
 
 
-=======================================================================
-            Imapsync tips for Exchange 20xx and Office365.
-=======================================================================
+
+======================================================================
+      Imapsync tips for Exchange 2000 2003 2007 2010 2013 2016 2019
+======================================================================
 
 Questions answered in this FAQ are:
 
-Q. Can I use imapsync to transfer from or to Exchange or Office365 accounts?
+Q. Can I use imapsync to transfer from or to Exchange accounts?
 
-Q. How to sync from XXX to Exchange 2010/2013/2016
-
-Q. How to sync from Office365 to XXX?
-
-Q. How to sync from XXX to Office365
-
-Q. For Office365 I have double and triple checked the username and 
-   password spelling but I still get a "LOGIN failed". Any clue?
+Q. How to sync from XXX to Exchange 2010/2013/2016/2019
 
 Q. I see "NO Maximum size of appendable message has been exceeded"
-   What can I do with that? Happens with Office365 or Exchange 2016.
+   What can I do with that?
 
-Q. Office365 throttles the sync and says: 
+Q. Exchange throttles the sync and says: 
    "Request is throttled. Suggested Backoff Time: 299961 milliseconds".
    What can I do with that?
 
-Q. Office365 refuses to create the folder named "Files" with the error
-   "NO Folder name is reserved". What happens?
-
-Q. Office365 users complain that a folder named "Files" contains 
-   messages with no sender.
+Q. For Exchange I have double and triple checked the username and
+   password spelling but I still get a "LOGIN failed". Any clue?
 
 Q. Exchange fails with "User is authenticated but not connected".
 
 Q. Exchange fails with "BAD Command received in Invalid state".
 
-Q. From XXX to Exchange 2013 or Office365, read receipts are all
-   resent again after a sync. Even for old messages. How can I fix that?
+Q. Exchange fails with "NO Server Unavailable. 15" and imapsync says
+   "could not append ... NO Server Unavailable. 15"
+
+Q. From XXX to Exchange 2013, read receipts are all resent again 
+   after a sync. Even for old messages. How can I fix that?
 
 Q. DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error
    DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error   
 
-Q. From XXX to Exchange 2010/2013 or Office365 I get this error message
+Q. From XXX to Exchange 2010/2013 I get this error message
    sometimes: "BAD Command Argument Error 11". What does it mean?
 
-Q. From XXX to Exchange 2010 or 2013 or Office365 the flag Flagged does 
+Q. From XXX to Exchange 2010 or 2013, the flag Flagged does 
    not seem to be well synced. What can I do?
 
-Q. Exchange and Office365 have throttle mechanisms to limit any huge 
-   usage. Sometimes imapsync transfers are too stressful for servers. 
-   How to deal with that?
-
-Q. How to migrate from or to Exchange 2007/2010/2013 with an 
+Q. How to migrate from or to Exchange 2007/2010/2013 with an
    admin/authuser account?
 
 Q. How to migrate from or to Exchange 2016 with an admin/authuser account?
 
-Q. How to migrate from or to Office 365 with an admin/authuser account?
-
-Q. How to migrate from or to Exchange 2003 with an admin/authuser 
-   account?
+Q. How to migrate from or to Exchange 2003 with an admin/authuser account?
 
 Q. Couldn't create folder [trash] "Mailbox already exists".
 
 Q. Migrating to Exchange 201O, messages get date of the transfer,
    this is bad for sorting and listing. What can I do?
 
-Q. How to sync from any to Exchange2007?
+Q. How to sync from any to Exchange 2007?
+
+Q. Exchange 2007 folders ending with a space cause an imapsync crash.
+   "Could not select: socket closed while reading data from server"
 
 Q. How to sync from Microsoft Exchange 2000 IMAP4rev1 server?
 
 
 Now the questions again with their answers.
 
-=======================================================================
-Q. Can I use imapsync to transfer from or to Exchange or Office365 accounts?
+======================================================================
+Q. Can I use imapsync to transfer from or to Exchange accounts?
 
-R. Yes. But IMAP access to a Exchange or Office365 account is not always 
+R. Yes. But IMAP access to an Exchange account is not always 
    allowed by default so it has to be allowed in the server configuration 
    part.
 
-   Exchange:  https://technet.microsoft.com/en-us/library/bb124489(v=exchg.150).aspx
-   Office365: https://support.microsoft.com/en-us/help/2416434/how-to-enable-or-disable-pop3-imap-mapi-outlook-web-app-or-exchange-ac
+   Exchange:  https://docs.microsoft.com/en-us/exchange/enable-imap4-in-exchange-2013-exchange-2013-help
 
-=======================================================================
-Q. How to sync from XXX to Exchange 2010/2013/2016
+======================================================================
+Q. How to sync from XXX to Exchange 2010/2013/2016/2019
 
-R0. IMAP is not enable by default on Exchange, see how to enable it:
-https://technet.microsoft.com/en-us/library/bb124489(v=exchg.150).aspx
+R0. IMAP is not enabled by default on Exchange, see how to enable it:
+https://docs.microsoft.com/en-us/exchange/enable-imap4-in-exchange-2013-exchange-2013-help
+Also read
+http://clintboessen.blogspot.com/2018/03/binding-certificate-breaks-imap-or-pop.html
 
 
 R1. Following is a command line resume that solves most encountered 
@@ -104,107 +95,100 @@ On Windows, use:
 
   imapsync.exe ... --exchange2
 
-or use (before imapsync release 1.836):
+which is equivalent in imapsync release 1.970 to:
 
   imapsync.exe ... ^
-    --maxsize 10000000 ^
+    --maxsize 10_000_000 ^
     --maxmessagespersecond 4 ^
-    --regexflag "s/\\Flagged//g" ^
     --disarmreadreceipts ^
-    --regexmess "s,(.{9900}),$1\r\n,g"
+    --regexflag "s/\\Flagged//g" ^
+    --regexmess "s,(.{10239}),$1\r\n,g"
 
 On Unix, use:
 
   imapsync ... --exchange2
 
-or use (before imapsync release 1.836):
+which is equivalent in imapsync release 1.970 to:
 
   imapsync ... \
-    --maxsize 10000000 \
-    --maxlinelength 9900 \
+    --maxsize 10_000_000 \
     --maxmessagespersecond 4 \
+    --maxlinelength 10239 \
     --regexflag 's/\\Flagged//g' \
     --disarmreadreceipts \
-    --maxlinelengthcmd 'reformime -r7' 
-    
-On Linux, to get the "reformime" command, install the "maildrop" package.
-In case you don't have it you can still use 
-  --regexmess 's,(.{9900}),$1\r\n,g'
-instead of --maxlinelengthcmd 'reformime -r7' 
-
-=======================================================================
-Q. How to sync from Office365 to XXX?
-
-R. On Windows, use:
-
-  imapsync.exe ... --office1
-
-On Unix, use:
-
-  imapsync ... --office1
+    --regexmess 's,(.{10239}),$1\r\n,g'
 
 
-=======================================================================
-Q. How to sync from XXX to Office365
 
-R0. IMAP is not enable by default on Office365, see how to enable it:
-https://support.microsoft.com/en-us/help/2416434/how-to-enable-or-disable-pop3-imap-mapi-outlook-web-app-or-exchange-ac
-https://technet.microsoft.com/en-us/library/gg298947(v=exchg.150).aspx
-https://support.context.io/hc/en-us/articles/213586243-Enabling-IMAP-for-Office365
+On Linux, you can also try the "reformime" command 
+that can be used like:
 
-R. Here is a command line resume that solves most encountered issues when 
-   migrating to Office365. It's similar with Exchange except for some 
-   values. To understand or change the details you have to read 
-   next Q/R sections.
+  imapsync ... --maxlinelengthcmd "reformime -r7"
 
+To get reformime, install the "maildrop" package.
 
-On Windows, use:
+On Linux again, there is a good Python script in the tarball that can 
+fix several things that Exchange or O365 have issues with.
 
-  imapsync.exe ... --office2
+Use it like this:
 
-or use (before imapsync release 1.836):
-  imapsync.exe ... ^
-    --maxsize 45000000 ^
-    --maxmessagespersecond 2 ^
-    --regexflag "s/\\Flagged//g" ^
-    --disarmreadreceipts ^
-    --regexmess "s,(.{10500}),$1\r\n,g"
+  ./imapsync ... --pipemess W/tools/fix_email_for_exchange.py
 
-On Unix, use:
+It often does some miracles on messages.
 
-  imapsync ... --office2
-
-or use (before imapsync release 1.836):
-
-  imapsync ... \
-    --maxsize 45000000 \
-    --maxlinelength 10500 \
-    --maxmessagespersecond 2 \
-    --regexflag 's/\\Flagged//g' \
-    --disarmreadreceipts \
-    --maxlinelengthcmd 'reformime -r7' 
-    
-On Linux, to get the "reformime" command, install the "maildrop" package.
-In case you don't have it you can use 
-  --regexmess 's,(.{10500}),$1\r\n,g'
-instead of --maxlinelengthcmd 'reformime -r7' 
-
-=======================================================================
-Q. For Office365 I have double and triple checked the username and 
+======================================================================
+Q. For Exchange I have double and triple checked the username and
    password spelling but I still get a "LOGIN failed". Any clue?
 
-R1. Triple check the hostname then. Try all of these:
- * imap-mail.outlook.com
- * imap.outlook.com
- * outlook.office365.com
 
-R2. Also triple check a license is assigned to that account 
-    in Office365.
+R1. Disable double-step authentication, also known as 2-factor,
+    2-step authentication on the Azure/Active Directory portal.
 
-R3. Try with a classic email client like Thunderbird and the same
-    parameters.   
+R2. Microsoft introduced something called "security defaults" which is
+enabled by default for new tenants. One of the rules blocks IMAP
+access as of imapsync.  The funny thing is that you can't disable
+single rules of this security package without buying additional
+licenses.  Switching the whole thing off allows the login.
 
-=======================================================================
+Also, disable double-step authentication on the Azure/Active Directory
+portal.
+
+See here:
+https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/concept-fundamentals-security-defaults
+Thanks to Stephan Buhre for this R2 answer.
+
+R3. I want to migrate to Exchange 2019. I haven't been able to sync
+using admin credentials even if I've tried and made all the
+configuration mentioned in the FAQ.Exchange.
+The error was always "LOGIN failed".
+
+My problem was that I have been used an admin account which haven’t
+attached a mailbox in Exchange. This is my admin account which I use
+only for configuration in Exchange, not for emails.
+
+So, after I’ve created in Exchange a mailbox for this user, the sync
+started to work.  I can confirm that imapsync is working for Exchange
+2019, with plain authentication, and full permissions of admin account
+(this account must have mailbox attached) applied to regular user
+mailbox account.
+
+It doesn't work with the --authuser2 parameter. The syntax that
+works for me is:
+
+  imapsync ... --host2 192.168.xxx.yyy  --user2 domain.local\admin.account\regular.user --password2 "xxxxxxx"
+or 
+  imapsync ...--host2 192.168.xxx.yyy  --user2 admin.account@mycompany.com\regular.user --password2 "xxxxxxx" 
+
+Thanks to Ioan Nechita for this R3 answer.
+
+R4. Are there special characters in the password?
+    https://imapsync.lamiral.info/FAQ.d/FAQ.Authentication_failure.txt
+    https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Windows.txt
+    https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt
+    https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Mac.txt
+
+
+======================================================================
 Q. Exchange fails with "User is authenticated but not connected".
 
 R1. "The message User is authenticated but not connected is due to a 
@@ -212,7 +196,7 @@ R1. "The message User is authenticated but not connected is due to a
    presents a valid user name but an invalid password, the server 
    accepts the login, but subsequent commands fail with the 
    aforementioned error message." Source:
-http://unix.stackexchange.com/questions/164823/user-is-authenticated-but-not-connected-after-changing-my-exchange-password
+https://unix.stackexchange.com/questions/164823/user-is-authenticated-but-not-connected-after-changing-my-exchange-password
 Thanks to James Abbottsmith for this link and explanation at
 https://github.com/imapsync/imapsync/issues/32#issuecomment-153561647
 
@@ -220,34 +204,26 @@ R2. Miguel Alameda reported understanding and solving this issue
     like this, the context was admin/authuser:
     "The admin user had not permission in the target mailbox."
 
-=======================================================================
-Q. I see "NO Maximum size of appendable message has been exceeded"
-   What can I do with that? Happens with Office365 or Exchange 2016
+R3. Jason from magiclamp.net adds this tip (2021_11_23).
+"In GoDaddy Exchange admin center, under recipients, you can 
+double click on a mailbox, and in the "mailbox delegation" scrolling 
+down to the bottom it shows a input field named "Full Access". 
+Add the admin account there."
 
-R0. It happens with Office365 and Exchange 2016.
-    See R1 for Office365
-    See R2 for Exchange 2016
 
-R1. Office365 supports send/receive max message sizes of up to 150MB 
-   but you need to make changes in your tenant(s) to support it.
 
-The following PowerShell command will increase the message sizes that 
-can be sent/received.  The trick in getting IMAPSync to work is to 
-apply these settings to the accounts performing the migration,
-NOT the accounts associated with the target mailbox (assuming you're
-using service accounts to perform transfers on behalf of users).
+======================================================================
+Q. I see "NO Maximum size of appendable message has been exceeded" 
+   What can I do with that?
 
-  Set-mailbox -Identity $UPN -MaxReceiveSize 150mb -MaxSendSize 150mb
+R1.  With Exchange 2019
 
-e.g. 
+Change the recipient Receive/Send size:
 
-  Set-mailbox -Identity "migrationaccount@testtenant.onmicrosoft.com" -MaxReceiveSize 150mb -MaxSendSize 150mb
+  Set-mailbox -Identity user@contoso.com -MaxReceiveSize 100mb -MaxSendSize 100mb
 
-We're transferring data between Office 365 tenants so we set these 
-values on the migration acounts in the source and target tenants.
+Thanks to Pasternak Denis for this Exchange 2019 tip.
 
-Thanks to Sean McDougall, Ian Thomas & Matt Wilks from Toronto 
-for this FAQ item.
 
 R2. With Exchange 2016
 
@@ -260,8 +236,8 @@ for each connector (with exchange admin panel), for each Postfach
 (using exchange management shell).
 
 For details and instructions, check 
-https://technet.microsoft.com/en-us/library/bb124345(v=exchg.160).aspx
-https://technet.microsoft.com/en-us/library/hh529949(v=exchg.160).aspx
+https://docs.microsoft.com/en-us/Exchange/mail-flow/message-size-limits?view=exchserver-2019
+https://docs.microsoft.com/en-us/Exchange/architecture/client-access/client-message-size-limits?view=exchserver-2019
 
 Unfortunately, the second link does not give instructions to remove the limit from imap4 service.
 
@@ -276,27 +252,28 @@ Please note that this change may be lost each time you install an
 cumulative update. After the change, restart the services exchange 
 transport role, exchange information store and exchange imap4 Backend.
 
-(Source: https://social.technet.microsoft.com/Forums/de-DE/2c07cf43-fe0e-4d22-b061-676a67c6d58d/exchange-2013-max-mailgre-per-imap?forum=exchange_serverde)
+Source in Deutch: 
+https://social.technet.microsoft.com/Forums/de-DE/2c07cf43-fe0e-4d22-b061-676a67c6d58d/exchange-2013-max-mailgre-per-imap?forum=exchange_serverde
 
 This change made our Exchange installation accept bigger e-mails 
 in order to remove the imapsync Exchange error message
 "NO Maximum size of appendable message has been exceeded"
 
 
-=======================================================================
-Q. Office365 throttles the sync and says: 
+======================================================================
+Q. Exchange throttles the sync and says: 
    "Request is throttled. Suggested Backoff Time: 299961 milliseconds".
    What can I do with that?
-   
-R. To solve the throttles issues from 0365, there's two solutions:
 
-R1. Call Microsoft Office365 and ask them to remove the limits on your 
-  mailboxes. That's not a joke, they do it for 90 days usually, 
-  sometimes only after you reach the second technician you call, 
-  the first one been not enough competent to understand what you're
-  talking about.
+R0. Exchange has throttle mechanisms to limit any huge 
+   usage. Sometimes imapsync transfers are too stressful for servers.
+   The message 
+   "Request is throttled. Suggested Backoff Time: 299961 milliseconds"
+   comes from the imap Office365 server, imapsync just reports it
+   before being disconnected from it.
 
-R2. Play with options --maxbytespersecond or --maxmessagespersecond
+
+R1. Play with options --maxbytespersecond or --maxmessagespersecond
     or --exitwhenover
 
   imapsync ... --maxbytespersecond 100_000
@@ -308,40 +285,17 @@ R2. Play with options --maxbytespersecond or --maxmessagespersecond
 I don't know the upper value that avoid the default throttling from 0365
 and I guess it changes over time.
 
-=======================================================================
-Q. Office365 refuses to create the folder named "Files" with the error
-   "NO Folder name is reserved". What happens?
+R2. For Exchange, in case throttle appears anyway, fix them with:
+  https://docs.microsoft.com/en-us/exchange/change-user-throttling-settings-for-specific-users-exchange-2013-help
+  See also:
+  http://linux-france.tk/prj/imapsync_list/msg02072.html 
+  (Subject: Re: [imapsync] From Exchange to Dovecot. Speed
+   Date: Mon, 03 Nov 2014 00:14:59 +0100)
 
-R. The folder Files is a standard folder in Office365. It should not
-   be synced in IMAP. See the next question.
+  Sometimes restarting the Exchange server is needed to take
+  into account the change in the configuration.
 
-=======================================================================
-Q. Office365 users complain that a folder named "Files" contains 
-   messages with no sender.
-
-R0. To fix this, add --exclude Files
-
-  imapsync ... --exclude Files
-
-R1. This folder "Files" seems to be a standard folder in Exchange Online, 
-   but it is not. The folder contains all attachments in every email 
-   that is in the mailbox, but without any headers.
-   This causes some confusion for users as these appear in their 
-   search results as duplicate lines but without the sender details 
-   or even the message body.
-
-   This folder seems to be usually hidden so IMAP clients can’t see it, 
-   but for some reason sometimes it becomes visible.
-
-R2. It looks to be a common problem with Exchange Online. 
-    I’m not sure what causes the folder to appear.
-
-   More info here:
-https://answers.microsoft.com/en-us/msoffice/forum/msoffice_o365admin-mso_exchon-mso_o365b/exclude-the-exchange-online-system-folder-called/2adbdf84-db4a-4c7f-ac29-738757980a0d
-https://answers.microsoft.com/en-us/msoffice/forum/msoffice_o365admin-mso_dep365-mso_o365b/no-sender-emails-in-files-folder/534bae8f-a7d7-4f5f-8ed7-5bad0d5fa23f
-
-  (This question/answer is taken quasi verbatim from Perttu Aaltonen)
-=======================================================================
+======================================================================
 Q. Exchange fails with "BAD Command received in Invalid state".
 
 R1. This message might happen when authenticating without ssl nor tls,
@@ -355,9 +309,38 @@ R2. This message might also happen when authenticating with ssl and tls.
    It won't work, you can't do a direct double encryption, few 
    servers allow that.
 
-=======================================================================
-Q. From XXX to Exchange 2013 or Office365, read receipts are all
-   resent again after a sync. Even for old messages. How can I fix that?
+======================================================================
+Q. Exchange fails with "NO Server Unavailable. 15" and imapsync says
+   "could not append ... NO Server Unavailable. 15"
+
+R1. The problem could be a mailbox quota limitation.
+
+In the Exchange log file, if you see many
+"Microsoft.Exchange.Data.Storage.QuotaExceededException",
+it looks like "Quota Exceeded" could be the issue.
+
+The solution is to increase the quota for this mailbox.
+Quoting
+https://docs.microsoft.com/en-us/exchange/recipients/user-mailboxes/storage-quotas?view=exchserver-2019
+In the Exchange admin center (EAC), navigate to Recipients > Mailboxes.
+In the list of user mailboxes, click the mailbox that you want to change the storage quotas for, 
+and then click Edit icon, then change "Prohibit send and receive at (GB)", then save your changes.
+
+Or use the Exchange Management Shell, aka PowerShell:
+Set-Mailbox -Identity "Joe Healy" -IssueWarningQuota 499gb -ProhibitSendQuota 500gb -ProhibitSendReceiveQuota 500gb -UseDatabaseQuotaDefaults $false
+
+
+R2. A user fixed it by enabling IMAP logging on Exchange which is
+    quite crazy as a solution. But why not, here it is:
+
+https://www.mantisbt.org/forums/viewtopic.php?p=68835&sid=8b1ecd2d976e85218fbf33fcd38c209d#p68835
+by TerranceM
+
+  Set-ImapSettings -Server <server-name> -ProtocolLogEnabled $true
+
+======================================================================
+Q. From XXX to Exchange 2013, read receipts are all resent again 
+   after a sync. Even for old messages. How can I fix that?
 
 R. imapsync can remove the header containing this read receipt request.
 
@@ -372,7 +355,7 @@ Selecting seen message can be done with:
 
   imapsync ... --search1 "SEEN" --disarmreadreceipts
 
-Of course a second run has to be run without the --disarmreadreceipts
+Of course a second run has to be done without the --disarmreadreceipts
 for unseen messages:
 
   imapsync ... --search1 "UNSEEN" 
@@ -389,7 +372,7 @@ X-Disposition-Notification-To: blabla
 
 Thanks to David Karnowski for pointing and solving this issue.
 
-=======================================================================
+======================================================================
 Q. DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error
    DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error   
 
@@ -405,8 +388,8 @@ R2. Another solution is to remove --tls or --ssl options for Exchange
 R3. See also the FAQ FAQ.SSL_errors.txt
     https://imapsync.lamiral.info/FAQ.d/FAQ.SSL_errors.txt
 
-=======================================================================
-Q. From XXX to Exchange 2010/2013 or Office365 I get this error message
+======================================================================
+Q. From XXX to Exchange 2010/2013 I get this error message
    sometimes: "BAD Command Argument Error 11". What does it mean?
 
 R. This error message comes from Exchange IMAP server when it 
@@ -414,17 +397,14 @@ R. This error message comes from Exchange IMAP server when it
 
   * Some messages are bigger than the size limit. 10 MB by default
     on Exchange. It can be upped by configuration for Exchange.
-	Sometimes a reboot is needed to take configuration changes 
-	into account.
+    Sometimes a reboot is needed to take configuration changes 
+    into account.
     If you can't fix this limit on Exchange then use option
     --maxsize 10000000 for 10 MB, change it if needed) to tell
     imapsync to skip those messages.
-    This value is 45 MB by default for Office365
 
   imapsync ... --maxsize 10000000 # 10 MB for Exchange
 
-  imapsync ... --maxsize 45000000 # 45 MB for Office365
-  
 
   * Quota reached. The whole account is full.
     It can be upped by configuration.
@@ -432,15 +412,10 @@ R. This error message comes from Exchange IMAP server when it
   * Some messages have some lines too long. Use option --maxlinelength
     to skip messages whose max line length is over a number of bytes.
     --maxlinelength 1000 is a RFC2822 must but most server support
-    higher values. 
-    Exchange supports 9900 characters line length:
+    higher values. Exchange supports 9900 characters line length:
 
   imapsync ... --maxlinelength 9900 
 
-    Office365 supports 10500 characters line length:
-
-  imapsync ... --maxlinelength 10500
-
   In case you prefer fixing messages with long lines the hard way,
   instead of skipping them with --maxlinelength 9900, just use:
 
@@ -452,11 +427,20 @@ R. This error message comes from Exchange IMAP server when it
   
 Have also in mind that Exchange closes the connection after 10 errors
 encountered so you might also see "BYE Connection closed" errors from
-Exchange, which means Exchange leaves the session and say goodbye,
+Exchange, which means Exchange leaves the session and says goodbye,
 come back later. Rerun a sync then.
 
-=======================================================================
-Q. From XXX to Exchange 2010 or 2013 or Office365 the flag Flagged does 
+On Linux, there is a good Python script in the tarball that can 
+fix several things that Exchange have issues with.
+
+Use it like this:
+
+  ./imapsync ... --pipemess W/tools/fix_email_for_exchange.py
+
+It often does some miracles on messages.
+
+======================================================================
+Q. From XXX to Exchange 2010 or 2013, the flag Flagged does 
    not seem to be well synced. What can I do?
 
 R. Use the following trick. Run imapsync twice, one with --regexflag
@@ -478,33 +462,14 @@ Thanks to Dave Murray and Simon Savva for reporting and solving
 this issue.
 
 ======================================================================
-Q. Exchange and Office365 have throttle mechanisms to limit any huge 
-   usage. Sometimes imapsync transfers are too stressful for servers. 
-   How to deal with that?
-
-R. It looks like limiting 4 messages per second is enough to avoid 
-   the throttle limits.
-
-  imapsync ... --maxmessagespersecond 4
-
-In case throttle appears anyway, fix them with:
-https://technet.microsoft.com/en-us/library/jj863577(v=exchg.150).aspx
-See also:
-http://www.linux-france.org/prj/imapsync_list/msg02072.html
-
-Sometimes restarting the Exchange server is needed to take
-into account the change in the configuration.
-
-======================================================================
-Q. How to migrate from or to Exchange 2007/2010/2013 with an 
+Q. How to migrate from or to Exchange 2007/2010/2013 with an
    admin/authuser account?
    
 R. The tricks comes from Michele Marcionelli and Benjamin Priestman:
 
-This doesn't work:
+This may works:
   imapsync ... --user2 user2 --authuser2 admin2 --password2 adminpassword2 ...
-
-This might works:
+or
   imapsync ... --user2 "domain\admin2\user2" --password2 adminpassword2 ...
 or 
   imapsync ... --user2 "admin2@domain\user2" --password2 adminpassword2 ...
@@ -532,7 +497,10 @@ Explanation, Exchange knows a) about a username
 and b) about a mailNickname which can be different to the username. 
 As it seems you need to use the mailNickname (e.g. alias) defined 
 for user2, not the username itself.
-http://blogs.technet.com/b/exchange/archive/2004/03/31/105275.aspx
+https://techcommunity.microsoft.com/t5/exchange-team-blog/understanding-login-strings-with-pop3-imap/ba-p/610683
+https://web.archive.org/web/20191014050543/https://techcommunity.microsoft.com/t5/Exchange-Team-Blog/Understanding-login-strings-with-POP3-IMAP/ba-p/610683
+the second link has the same content, I give it in case the first disappears,
+thanks to archive.org!
 
 A little note from John Becker.
 After you pointed out that I had issues with authorization, I realized that
@@ -571,36 +539,24 @@ and again - don't use administrator@domain.com - as that will never
 work because IMAP LOGIN is disabled for that account apparently
 under all circumstances.
 
-======================================================================
-Q. How to migrate from or to Office 365 with an admin/authuser account?
+Note from Noxyron
+https://github.com/imapsync/imapsync/issues/136#issuecomment-543761463
 
-Note from Yago Torres Fernandez:
-(a working command using admin/authuser on host2 Office 365)
-
-  imapsync ... --authuser2 user_admin@domain.com --user2 user_to_be_migrated@domain.com  ^
-               --password2 XXXX --ssl2  ^
-
-but previously in Office365 you must do something like that, using powershell:
-
-  Add-MailboxPermission -identity user_to_be_migrated@domain.com -user user_admin@domain.com -accessrights fullaccess -inheritancetype all
-
-PLAIN authentication is the only way to go with --authuser1 for now.
-So don't use --authmech1 SOMETHING with --authuser1 admin_user,
-it will not work.
-Same behavior with the --authuser2 option.
-
-Note from Martin Paulucci:
-I had to remove the domain part for the user
-but not for the admin.  Example:
-
-  imapsync ... --authuser2 user_admin@domain.com --user2 user_to_be_migrated
-
-See also:
-http://www.linux-france.org/prj/imapsync_list/msg02203.html
+Hi guys. For a long time I could not start synchronization, I use admin rights 
+and the --authuser2 switch when I migrated from Mdaemon to Exchange2016. 
+I always got the error:
+"Host2 failure: Error login on [my.exchange.com] with user [mydomainuser] auth [PLAIN]: 2 NO AUTHENTICATE failed."
+In order for ImapSync to login to the user's mailbox using administrator rights,
+you need to give for the administrator the permissions to the mailbox of this
+user with the command:
+Add-MailboxPermission -Identity DomainUser -User DomainAdmin -AccessRights FullAccess -InheritanceType All -AutoMapping $false
+using Exchange Management Shell. 
+Without this, synchronization will not work. 
+Please, add this comment to the FAQ.Exchange.txt
+(done!)
 
 ======================================================================
-Q. How to migrate from or to Exchange 2003 with an admin/authuser 
-   account?
+Q. How to migrate from or to Exchange 2003 with an admin/authuser account?
 
 R. Thomas Edgar wrote the following
 
@@ -632,7 +588,7 @@ In case you are not aware:
   omitted in our case
 - Use --debugimap when testing initial connectivity, if necessary
 
-=======================================================================
+======================================================================
 Q. Couldn't create folder [trash] "Mailbox already exists".
 
 R. Some servers take care about character case in folder names, 
@@ -657,7 +613,7 @@ R. Some servers take care about character case in folder names,
    imapsync ...  --regextrans2 "s/^trash$/Trash/"
 
    If there are two folders Trash and trash on host1 then both
-   will be merge into only one Trash folder on host2. 
+   will be merged into only one Trash folder on host2. 
    In case option --delete2 is used the regextans2 above becomes
    mandatory, otherwise imapsync will sync messages from the
    first Trash and then delete them when syncing trash.
@@ -671,17 +627,18 @@ Q. Migrating to Exchange 201O, messages get date of the transfer,
    this is bad for sorting and listing. What can I do?
 
 R1. Be sure to have at least Exchange 2010 SP2 Rollup 5
-http://www.tribalchicken.com.au/15-technical/29-imapsync-exchange2010
+https://tribalchicken.net/imap-sync-issues-with-exchange-2010/
 
 R2. See also the Outlook side (thanks to Martin Hochreiter for this solution)
-http://www.howto-outlook.com/faq/archivenotworking.htm
 by changing Archive to sent/receive date
+https://www.howto-outlook.com/faq/archivenotworking.htm
+https://web.archive.org/web/20190914063435/http://www.howto-outlook.com/faq/archivenotworking.htm
 
 It's often shorter to change one server than thousands clients
 so R1 might be easier to do.
 
 ======================================================================
-Q. How to sync from any to Exchange2007?
+Q. How to sync from any to Exchange 2007?
 
 Several problems:
 - Big messages: increase the "send- and receive-connector" 
@@ -692,18 +649,27 @@ R. 2 solutions
 R1. With imapsync (only partial success)
 --skipheader 'Received' \
 --regexmess 's{\A(.*?(?!^$))^Date:(.*?)$}{$1Date:$2\nReceived: From; $2}gxms'
-Any user having time to spend to debug Exchange2007 with imapsync
+Any user having time to spend to debug Exchange 2007 with imapsync
 is welcome.
 
 R2. Other solution
 Two users succeeded by using "MS Transporter Suite" (which is closed
 expensive non-free software).
 
+======================================================================
+Q. Exchange 2007 folders ending with a space cause an imapsync crash.
+   "Could not select: socket closed while reading data from server"
+
+R. Using another imap tool like Thunderbird or Outlook, change the 
+   folder name to exclude the trailing space.
+
+See  https://github.com/imapsync/imapsync/issues/154
+
 ======================================================================
 Q. How to sync from Microsoft Exchange 2000 IMAP4rev1 server?
 
 R. imapsync ...  --prefix1 "INBOX."
 
 
-=======================================================================
-=======================================================================
+======================================================================
+======================================================================

FAQ.d/FAQ.FirstClass.txt

@@ -0,0 +1,68 @@
+#!/bin/cat
+$Id: FAQ.FirstClass.txt,v 1.1 2021/01/26 14:03:58 gilles Exp gilles $
+
+This document is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.FirstClass.txt
+
+
+=======================================================================
+         Imapsync tips for the FirstClass IMAP server.
+=======================================================================
+
+
+
+======================================================================
+Q. From FirstClass to XXX
+   http://www.firstclass.com/
+
+R. Migrating from FirstClass is not easy because FirstClass, strangely, 
+   does not show all messages via IMAP. To make it show all messages,
+   a trick, painful to follow by hand, is moving emails 
+   out and back in, for each folder. May be it can be done by a script.
+
+  FirstClass releases prior to release 12 do not shows the "Sent" 
+  folder in IMAP but FirstClass release 12 shows it.
+  I advice you to upgrade to FirstClass release 12 before leaving it
+  with imapsync or another imap tool.
+
+Here is a command line used to migrate from FirtClass release 12:
+
+  imapsync ... \
+    --tmpdir /var/tmp --usecache \
+    --useheader Message-ID \
+    --idatefromheader \
+    --addheader \
+    --regextrans2 "s,(/|^) +,\$1,g" --regextrans2 "s, +(/|$),\$1,g" \
+    --regextrans2 "s/[\^]/_/g"       \
+    --regextrans2 "s/['\"\\\\]/_/g"  \
+    --regextrans2 "s,&AC8-,-,g" \
+    --regextrans2 "s,&APg-,oe,g" 
+
+On Windows:
+  imapsync.exe ... ^
+    --automap ^
+    --usecache ^
+    --useheader Message-ID ^
+    --idatefromheader ^
+    --addheader ^
+    --regextrans2 "s,(/|^) +,$1,g" ^
+    --regextrans2 "s, +(/|$),$1,g" ^
+    --regextrans2 "s/[\^]/_/g" ^
+    --regextrans2 "s/['\\]/_/g" ^
+    --regextrans2 "s,^&AC8-,-,g"  ^
+    --regextrans2 "s,^&APg-,oe,g"
+
+
+Special thanks to Kristian Wind and Joey Alexander for helping me
+writing this FAQ item. 
+See also this worth reading discussion in a Zimbra forum:
+https://web.archive.org/web/20120627043731/http://www.zimbra.com/forums/migration/20349-help-needed-migrating-firstclass.html
+
+Another advice with FirstClass as host1, do not use --delete1emptyfolders since
+it will destroy all subfolders of any empty folder. This is a FirstClass bug:
+when asked to list a non-empty folder it says it has no children while it does
+have ones.
+See the details and explanations at
+https://github.com/imapsync/imapsync/issues/207
+

FAQ.d/FAQ.Flags.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Flags.txt,v 1.20 2019/02/14 16:01:41 gilles Exp gilles $
+$Id: FAQ.Flags.txt,v 1.28 2022/06/22 17:07:35 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -22,6 +22,9 @@ Q. Exchange sends an email to any sender whose email is deleted
    without reading. It's called "unread notifications".
    How to set the \Seen flag on host1 (source system) before syncing?
 
+Q. Every single mail synced to exchange online owns the 
+   category $MDNSent after migration. How can I avoid this?
+
 Q. Does imapsync retain the \Answered and $Forwarded flags?
 
 Q. How to fix this error: BAD Invalid system flag \FORWARDED
@@ -41,6 +44,8 @@ Q. Is it possible to sync labels and stars made by Thunderbird to
    Exchange categories? Or a way in Outlook to show labels created by
    Thunderbird?
 
+Now the questions again with their answers.
+
 =======================================================================
 Q. How to debug flag issues?
 
@@ -149,22 +154,22 @@ Add the \Seen flag to all messages like this:
 
 On Winwows:
 
-  imapsync.exe ... --regexflag "s,^((?!\\Seen)).*$,$1 \\Seen,"
+  imapsync.exe ... --regexflag "s,\\Seen,," --regexflag "s,,\\Seen ,"
+                   
+On Unix:
+
+  imapsync ... --regexflag 's,\\Seen,,' --regexflag 's,,\\Seen ,'
+
+R2. You can also filter with --search1 UNSEEN and use only one
+    of the regexes:
 
 On Unix:
 
-  imapsync ... --regexflag 's,^((?!\\Seen)).*$,$1 \\Seen,' 
-
-R2. You can also filter with --search1 UNSEEN and use a simpler
-    regex:
+  imapsync ... --search1 UNSEEN --regexflag 's,,\\Seen ,'
 
 On Winwows:
 
-  imapsync.exe ... --search1 UNSEEN --regexflag "s/(.*)/$1 \\Seen/"
-
-On Unix:
-
-  imapsync ... --search1 UNSEEN --regexflag 's/(.*)/$1 \\Seen/'
+  imapsync.exe ... --search1 UNSEEN  --regexflag "s,,\\Seen ,"
 
 R3. Fix it on the server Exchange:
 Google translate: 
@@ -172,18 +177,30 @@ https://translate.google.com/translate?sl=auto&tl=en&u=https%3A%2F%2Fwww.ci-solu
 German original:
 https://www.ci-solution.com/blog/artikel/ungelesen-geloescht-verhindern.html
 (Link from Oliver B.)
+
+======================================================================
+Q. Every single mail synced to exchange online owns the 
+   category $MDNSent after migration. How can I avoid this?
+
+R. To remove the flag $MDNSent from all messages, use:
+
+  imapsync ... --regexflag "s/\$MDNSent//g"
+
+See also the document
+https://imapsync.lamiral.info/FAQ.d/FAQ.Flags.txt
+
 =======================================================================
 Q. Does imapsync retain the \Answered and $Forwarded flags?
 
 R. It depends on the destination server.
 
-a) If the destination server honours the "PERMAENTFLAGS \*" 
-directive (meaning it accepts any flag) or no PERMAENTFLAGS at all 
+a) If the destination server honours the "PERMANENTFLAGS \*" 
+directive (meaning it accepts any flag) or no PERMANENTFLAGS at all 
 then imapsync synchronizes all flags except the \Recent flag
 (RFC 3501 says about \Recent flag "This flag can not be 
 altered by the client.").
 
-b) If the destination server honours the "PERMAENTFLAGS without the 
+b) If the destination server honours the "PERMANENTFLAGS without the 
 special "\*"  then imapsync synchronizes only the flags listed 
 in PERMANENTFLAGS.
 
@@ -205,7 +222,7 @@ On Unix:
 
   imapsync ... --regexflag 's/\\FORWARDED//gi' 
 
-or
+or (on Unix)
 
   imapsync ... --regexflag "s/\\\\FORWARDED//gi" 
 
@@ -217,11 +234,18 @@ Unix:      imapsync ...     --regexflag 's/\\Indexed//gi'
 Windows:   imapsync.exe ... --regexflag "s/\\ATTACHED//gi" 
 Unix:      imapsync ...     --regexflag 's/\\ATTACHED//gi' 
 
-All usually problematic flags in one line:
+Removing usually problematic flags in one line:
 
-Windows:   imapsync.exe ... --regexflag "s/\\FORWARDED|\\Indexed|\\ATTACHED//gi" 
-Unix:      imapsync ...     --regexflag 's/\\FORWARDED|\\Indexed|\\ATTACHED//gi' 
+Windows:   imapsync.exe ... --regexflag "s/\\FORWARDED|\\Indexed|\\ATTACHED|\\RECEIPTCHECKED//gi" 
+Unix:      imapsync     ... --regexflag 's/\\FORWARDED|\\Indexed|\\ATTACHED|\\RECEIPTCHECKED//gi' 
 
+Removing all problematic flags I've seen so far in one line:
+Windows:   imapsync.exe ... --regexflag "s/\\RECEIPTCHECKED|\\Indexed|\\X-EON-HAS-ATTACHMENT|\\UNSEEN|\\ATTACHED|\\X-HAS-ATTACH|\\FORWARDED|\\FORWARD|\\X-FORWARDED|\\\$FORWARDED|\\PRIORITY|\\READRCPT//g"
+Unix:      imapsync     ... --regexflag 's/\\RECEIPTCHECKED|\\Indexed|\\X-EON-HAS-ATTACHMENT|\\UNSEEN|\\ATTACHED|\\X-HAS-ATTACH|\\FORWARDED|\\FORWARD|\\X-FORWARDED|\\\$FORWARDED|\\PRIORITY|\\READRCPT//g'
+
+Removing all flags but the standard ones from the IMAP RFC:
+Windows:   imapsync.exe ... --regexflag "s/.*?(?:(\\(?:Answered|Flagged|Deleted|Seen|Draft)\s?)|$)/defined($1)?$1:q()/eg"
+Unix:      imapsync     ... --regexflag 's/.*?(?:(\\(?:Answered|Flagged|Deleted|Seen|Draft)\s?)|$)/defined($1)?$1:q()/eg'
 
 =======================================================================
 Q. How to convert flags with $ to \ character?
@@ -296,7 +320,8 @@ You can try --nofilterflags straightaway without the
 "other thunderbird" proposal.
 
 There is also the possibility to map flags across servers 
-with --regexflags
+with the option --regexflag. See some --regexflag examples 
+above. 
 
 
 =======================================================================

FAQ.d/FAQ.Folders_Mapping.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Folders_Mapping.txt,v 1.17 2018/11/20 10:26:18 gilles Exp gilles $
+$Id: FAQ.Folders_Mapping.txt,v 1.23 2021/04/28 23:42:04 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -11,11 +11,93 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Mapping.txt
              Imapsync tips about changing folders names.
 =======================================================================
 
-Folders names are by default reproduced identical except for
-the prefix and the separator which are automatically adapted
-for host2.
+Questions answered in this FAQ are:
 
-This document gives most examples with the powerful but complex
+Q. What is the imapsync default behaviour with folders?
+
+Q. What encoding imapsync uses to deal with folder names?
+
+Q. How can I encode folder names for imapsync without getting a headache?
+
+Q. What are the easy ways to change folder names?
+
+Q. What should I know before playing with the --regextrans2 option?
+
+Q. How to remove the string "INBOX." in the name of destination folders?
+
+Q. How to change only INBOX name to Inbox_Migrated?
+
+Q. How to sync all folders to a single INBOX folder?
+
+Q. How to sync all folders to a single WHOLE folder?
+
+Q. How to sync a complete account in a subfolder called FOO?
+
+Q. How to substitute all characters dot "." by underscores "_" ?
+
+Q. How to substitute all doublequotes " by underscores _ ?
+
+Q. How to substitute all characters * % . and # by underscores _ ?
+
+Q. How to change folder names like this:
+[mail/Sent Items] -> [Sent]
+[mail/Test]       -> [INBOX/Test]
+[mail/Test2]      -> [INBOX/Test2]
+
+Q. Is it possible to synchronize all messages from one server to 
+   another without recreating the folder structure and the target server.
+
+Q. I would like to move emails from InBox to a sub-folder called,
+   say "2010-INBOX" based on the date. Like all emails received in the
+   year 2010 should be moved to the folder called "2010-INBOX".
+
+Q. How to sync a complete account in a subfolder called FOO,
+   the hard way?
+
+Q. How to transform all folder names to lowercase?
+
+Q. How to transform all folder names to uppercase?
+
+Q. How to transform all folder names to lowercase, except the first 
+   character in uppercase?
+
+
+Now the questions again with their answers.
+
+
+=======================================================================
+Q. What is the imapsync default behaviour with folders?
+
+R. Imapsync syncs all folders, one by one, in alphanumeric order,
+   reproducing their name identical from the source account at host1 to 
+   the destination account at host2, except for the prefix and the 
+   separator which are automatically adapted for host2.
+
+=======================================================================
+Q. What encoding imapsync uses to deal with folder names?
+
+
+R. The IMAP protocol has a specific way to code folders names,
+especially when these names use non-ascii 7bit characters.
+This encoding is called utf7imap. 
+
+Imapsync uses the same encoding as IMAP, it uses utf7imap.
+
+=======================================================================
+Q. How can I encode folder names for imapsync without getting a headache?
+
+
+R. In order to well specify folders names on the command line, imapsync
+   prints the complete folder list of both sides at the beginning 
+   of each run.
+
+The left column is the encoding you have to use, without the first enclosing 
+square brackets [], the right column is the human utf8 view.
+
+=======================================================================
+Q. What are the easy ways to change folder names?
+
+R. This document gives most examples with the powerful but complex
 option --regextrans2. Before using --regextrans2 you should 
 consider using --automap and --f1f2 because they are simpler 
 to understand and use.
@@ -33,22 +115,25 @@ to understand and use.
                        Add --debug to see what's really going on.)
 
 
+=======================================================================
+Q. What should I know before playing with the --regextrans2 option?
 
-Things to know and understand before playing with --regextrans2
+The things to know and understand before playing with --regextrans2 are:
 
 *) --regextrans2 is used to transform folders names.
 
 *) --regextrans2 applies after the default 
-   inversion prefix1 <-> prefix2 and sep1 <-> sep2
-   So when elaborating the regex you should focus on
+   inversion prefix1 <-> prefix2 and sep1 <-> sep2.
+   So, when elaborating the regex you should focus on
    the right part of the default mapping printed by
    imapsync, the part showing the host2 folder name.
-   The section to look at is within the folder loop:
+   The section to look at is either the folder size section
+   or within the folder loop located "Here" between the lines
 ++++ Looping on each folder
 Here
 ++++ End looping on each folder
-
-*) Several --regextrans2 is possible, they will be applied in the order
+   
+*) Several --regextrans2 are possible, they will be applied in the order
    of the command line, each one on the result of the previous one.
 
 *) --regextrans2 uses Perl regex mechanism so it may be hard to master
@@ -61,18 +146,19 @@ Here
 
    On Linux/Unix use single quotes ' around the regex string, it is
    easier to get what we want with single quotes since the shell 
-   won't change the inner string. Like --regextrans2 'myregex'
+   won't change the inner string. Use --regextrans2 'myregex'
 
 *) Good method to elaborate any --regextrans2 string
 
    First, elaborate the --regextrans2 string with --dry --justfolders options. 
 
-   imapsync ... --dry --justfolders
+     imapsync ... --dry --justfolders
 
    With --dry imapsync shows the transformations it will do without 
    really doing them, --dry is the "do nothing" mode.
    With --justfolders imapsync will work only with folders, 
-   messages won't be taken into account, so it will be fast.
+   messages won't be taken into account, so it will be fast
+   and focused on the folders names.
 
    When the output shows what you expect imapsync to do with folders
    names, you can remove the --dry option. Keep the --justfolders 
@@ -85,76 +171,60 @@ Here
    ETA calculation and it's a supplementary check on folders.
 
 =======================================================================
-Q. Give examples about --regextrans2
+Q. How to remove the string "INBOX." in the name of destination folders?
 
 
-Examples: 
+R. To remove "INBOX." in the name of destination folders:
 
-1) To remove INBOX. in the name of destination folders
+On Linux:
 
   imapsync ... --regextrans2 's/^INBOX\.(.+)/$1/'
 
-2) To change only INBOX to Inbox_Migrated
+On Windows:
 
+  imapsync ... --regextrans2 "s/^INBOX\.(.+)/$1/"
+
+=======================================================================
+Q. How to change only INBOX name to Inbox_Migrated?
+
+R. 
    imapsync ... --regextrans2 's{^INBOX$}{Inbox_Migrated}'
 
 
-2a) To sync all folders to INBOX
+=======================================================================
+Q. How to sync all folders to a single INBOX folder?
+
+R. 
 
   imapsync ... --regextrans2 "s/.*/INBOX/"
 
+=======================================================================
+Q. How to sync all folders to a single WHOLE folder?
 
-2b) To sync a complete account in a subfolder called FOO
+R. 
 
-Since imapsync release 1.641 simply use
+  imapsync ... --regextrans2 "s,.*,WHOLE,"
+
+
+=======================================================================
+Q. How to sync a complete account in a subfolder called FOO?
+
+R. With imapsync release 1.641 and former, simply use:
 
   imapsync ... --subfolder2 FOO
-  
-Next examples are  subfolder solutions for any release.
 
-  a) Separator is dot character "." and "INBOX" prefixes every folder
+Examples to this subfolder problem for previous releases
+can be seen below, in another Q/R section.
 
-On Linux/Unix:
 
-    --regextrans2 's,^INBOX(.*),INBOX.FOO$1,'
+=======================================================================
+Q. How to substitute all characters dot "." by underscores "_" ?
 
-On Windows:
-
-    --regextrans2 "s,^INBOX(.*),INBOX.FOO$1,"
-
-  or:
-  
-  b) Separator is the slash character "/" and there is no prefix 
-
-On Linux/Unix:
-
-   --regextrans2 's,(.*),FOO/$1,'
-
-On Windows:
-
-   --regextrans2 "s,(.*),FOO/$1,"
-   
-   or:
-
-  c) Any separator, any prefix solution, FOO is the subfolder:
-
-  It is a complicated line because every case is taken into account.
-  Type it in one line (or with the \ at the end of first line on Unix shells.
-
-On Linux/Unix:
-
-   --regextrans2 's,${h2_prefix}(.*),${h2_prefix}FOO${h2_sep}$1,' \
-   --regextrans2 's,^INBOX$,${h2_prefix}FOO${h2_sep}INBOX,'
-
-On Windows:
-
-  --regextrans2 "s,${h2_prefix}(.*),${h2_prefix}FOO${h2_sep}$1," ^
-  --regextrans2 "s,^INBOX$,${h2_prefix}FOO${h2_sep}INBOX,"
-
-3) to substitute all characters dot "." by underscores "_"
+R.
     --regextrans2 "s,\.,_,g"
 
-3b) to substitute all doublequotes " by underscores _
+=======================================================================
+Q. How to substitute all doublequotes " by underscores _ ?
 
 On Linux/Unix:
 
@@ -164,8 +234,8 @@ On Windows:
 
   --regextrans2 s,\^",_,g
 
-3c) to substitute all characters *%. by underscores _
-You can increase the *%. list by any unwanted character.
+=======================================================================
+Q. How to substitute all characters * % . and # by underscores _ ?
 
 On Linux/Unix:
 
@@ -175,12 +245,15 @@ On Windows:
 
   --regextrans2 "tr,*%.#,_,"
 
-3d) It is a bad idea to substitute & characters since &
+You can increase the *%.# list by any unwanted character,
+plus, the order doesn't count. See below for &
+
+It is a bad idea to substitute & characters since &
 is a character to encode non-ascii characters in IMAP folder names.
 
 
-
-4) to change folder names like this:
+=======================================================================
+Q. How to change folder names like this:
 [mail/Sent Items] -> [Sent]
 [mail/Test]       -> [INBOX/Test]
 [mail/Test2]      -> [INBOX/Test2]
@@ -190,12 +263,56 @@ On Linux/Unix:
     --regextrans2 's,^mail/Sent Items$,Sent,' \
     --regextrans2 's,^mail/,INBOX/,'
 
+a) I want folders 'Sent', 'Sent Messages', 'Gesendet' from host1 
+    to be merged to folder 'Sent' on host2.
+
+  --regextrans2 "s{Sent|Sent Messages|Gesendet}{Sent}"
+
+b) I do not want this rule a) to apply to any subfolders,
+    e.g. 'folder/Sent' or 'folder/Sent Messages' etc. but only
+    basefolders.
+
+So you have to add an anchor character ^ meaning "beginning of the string",
+and an anchor character $ meaning "end of the string" like this:
+
+  --regextrans2 's{^Sent$|^Sent Messages$|^Gesendet$}{Sent}'
+
+c) I want just a) to be combined with a subfolder, for example:
+
+  INBOX                -> Subfolder/INBOX
+  Sent                 -> Subfolder/Sent
+  Sent Messages        -> Subfolder/Sent
+  Gesendet             -> Subfolder/Sent
+  folder/Sent          -> Subfolder/folder/Sent
+  folder/Sent Messages -> Subfolder/folder/Sent
+  folder/Gesendet      -> Subfolder/folder/Sent
+
+To achieve this, use:
+
+  --subfolder2 Subfolder --regextrans2 's{Sent|Sent Messages|Gesendet}{Sent}'
+
+d) I want a) and b) to be combined with a subfolder, for example:
+
+  INBOX                -> Subfolder/INBOX
+  Sent                 -> Subfolder/Sent
+  Sent Messages        -> Subfolder/Sent
+  Gesendet             -> Subfolder/Sent
+  folder/Sent          -> Subfolder/folder/Sent
+  folder/Sent Messages -> Subfolder/folder/Send Messages
+  folder/Gesendet      -> Subfolder/folder/Gesendet
+  ...
+
+This one is trickier to achieve:
+
+  --subfolder2 Subfolder --regextrans2 's{^Subfolder/Sent$|^Subfolder/Sent Messages$|^Subfolder/Gesendet$}{Subfolder/Sent}'
+
 
 ======================================================================
 Q. Is it possible to synchronize all messages from one server to 
-another without recreating the folder structure and the target server.
+   another without recreating the folder structure and the target server.
 
-R. Yes.
+R. Yes. This question has already been answered but a little of 
+   redundancy can't hurt, can he?
 
 For example, to synchronize all messages in all folders on host1
 to folder INBOX only on host2:
@@ -214,39 +331,26 @@ to folder INBOX only on host2:
 4) Remove --justfolders
 
 
-======================================================================
-Q. I have moved from Braunschweig to Graz, so I would like to have my
-   whole Braunschweig mail sorted into a sub-folder INBOX.Braunschweig 
-   of my new mail account.
-
-R.
-1) First try (safe mode):
-
-imapsync \
- ...
- --regextrans2 "s/INBOX(.*)/INBOX.Braunschweig\$1/" \
- --dry --justfolders
-
-On Windows, in the previous example containing \$1 you have to 
-replace the two \$1 by $1 (remove the \ before $).
-
-
-2) See if the output says everything you want imapsync to do, 
-   --dry option is safe and does nothing real.
-   
-3) Remove --dry 
-   Check the imap folder tree on the target side
-
-4) Remove --justfolders
-
 =======================================================================
 Q. I would like to move emails from InBox to a sub-folder called,
-   say "2010-INBOX" based on the date (Like all emails received in the
-   Year 2010 should be moved to the folder called "2010-INBOX").
+   say "2010-INBOX" based on the date. Like all emails received in the
+   year 2010 should be moved to the folder called "2010-INBOX".
 
 R. 2 ways :
 
-a) Manually:
+a) With imapsync:
+-----------------
+
+  imapsync ... \
+    --search "SENTSINCE 1-Jan-2010 SENTBEFORE 31-Dec-2010"
+    --f1f2 "INBOX=2010-INBOX" \
+    --folder INBOX
+
+See also the FAQ FAQ.Messages_Selection.txt
+https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt
+to get more examples about how to select messages.
+
+b) Manually:
 ------------
 
 1) You create a folder INBOX.2010-INBOX
@@ -257,13 +361,130 @@ a) Manually:
 
 3) Cut/paste in INBOX.2010-INBOX
 
-b) With imapsync:
------------------
 
-imapsync ... \
---search 'SENTSINCE 1-Jan-2010 SENTBEFORE 31-Dec-2010'
---regextrans2 's/^INBOX$/INBOX.2010-INBOX/' \
---folder INBOX
+
+=======================================================================
+Q. How to sync a complete account in a subfolder called FOO,
+   the hard way?
+
+R0. See the easy above using --subfolder2
+
+R1. Several ways to do it.
+
+  a) Separator is dot character "." and "INBOX" prefixes every folder
+
+On Linux/Unix:
+
+    --regextrans2 's,^INBOX(.*),INBOX.FOO$1,'
+
+On Windows:
+
+    --regextrans2 "s,^INBOX(.*),INBOX.FOO$1,"
+
+  or:
+  
+  b) Separator is the slash character "/" and there is no prefix 
+
+On Linux/Unix:
+
+   --regextrans2 's,(.*),FOO/$1,'
+   
+   or with doublequotes
+   
+   --regextrans2 "s,(.*),FOO/\$1,"
+
+On Windows:
+
+   --regextrans2 "s,(.*),FOO/$1,"
+
+=======================================================================
+Q. How to transform all folder names to lowercase?
+
+R. 
+
+First, some documentation to understand how the regexes work.
+perldoc perlre says:
+
+  \l          lowercase next char (think vi)
+  \u          uppercase next char (think vi)
+  \L          lowercase till \E (think vi)
+  \U          uppercase till \E (think vi)
+  \E          end case modification (think vi)
+  \Q          quote (disable) pattern metacharacters till \E
+
+
+On Linux:
+
+If the separator on host2 is the character slash "/", use:
+
+  --regextrans2 's,([^/]+),\L$1\E,g'
+
+If the separator on host2 is the character dot ".", use:
+
+  --regextrans2 's,([^.]+),\L$1\E,g'
+
+On Windows:
+
+If the separator on host2 is the character slash "/", use:
+
+  --regextrans2 "s,([^/]+),\L$1\E,g"
+
+If the separator on host2 is the character dot ".", use:
+
+  --regextrans2 "s,([^.]+),\L$1\E,g"
+
+
+=======================================================================
+Q. How to transform all folder names to uppercase?
+
+R. 
+
+On Linux:
+
+If the separator on host2 is the character slash "/", use:
+
+  --regextrans2 's,([^/]+),\U$1\E,g'
+
+If the separator on host2 is the character dot ".", use:
+
+  --regextrans2 's,([^.]+),\U$1\E,g'
+
+On Windows:
+
+If the separator on host2 is the character slash "/", use:
+
+  --regextrans2 "s,([^/]+),\U$1\E,g"
+
+If the separator on host2 is the character dot ".", use:
+
+  --regextrans2 "s,([^.]+),\U$1\E,g"
+
+=======================================================================
+Q. How to transform all folder names to lowercase, except the first 
+   character in uppercase?
+
+R. 
+
+On Linux:
+
+If the separator on host2 is the character slash "/", use:
+
+  --regextrans2 's,([^/]+),\u\L$1\E,g'
+
+If the separator on host2 is the character dot ".", use:
+
+  --regextrans2 's,([^.]+),\u\L$1\E,g'
+
+On Windows:
+
+If the separator on host2 is the character slash "/", use:
+
+  --regextrans2 "s,([^/]+),\u\L$1\E,g"
+
+If the separator on host2 is the character dot ".", use:
+
+  --regextrans2 "s,([^.]+),\u\L$1\E,g"
+
 
 =======================================================================
 =======================================================================

FAQ.d/FAQ.Folders_Selection.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Folders_Selection.txt,v 1.7 2018/05/24 11:34:30 gilles Exp gilles $
+$Id: FAQ.Folders_Selection.txt,v 1.12 2022/03/22 11:12:32 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -10,16 +10,94 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Selection.txt
                Imapsync tips to select folders.
 =======================================================================
 
+Questions answered in this FAQ are:
+
+Q. What is the default imapsync behaviour about folders?
+
+Q. What are the options to select folders?
+
+Q. Is it ok to use --subfolder1 and --subfolder2 to just select folders?
+
+Q. How can I sync only one folder?
+
+Q. What are --subscribe and --subscribed for, and how can they be used?
+
+Q. How to exclude a folder hierarchy like "public"?
+
+Q. How to exclude only the INBOX folder?
+
+Q. How to exclude folders matching SPAM no matter the case, 
+   aka, how to be case insensitive?
+
+Q. How to make option --folder "MyFolder" be recursive?
+
+Q. How to change folders names or move them elsewhere?
+
+Now the questions again with their answers.
+
+=======================================================================
+Q. What is the default imapsync behaviour about folders?
+
 By default, Imapsync syncs all folders, one by one, in alphanumeric order.
 
-The IMAP protocol has a specific way to code folder names,
-when these names use non-ascii 7bit characters.
-Imapsync uses the same code as IMAP. 
-In order to well specify them on the command line you will be
-helped by the complete folder list printed by imapsync at each run.
-The left column is what you have to use without the first enclosing 
-square brackets, the right column is the human utf8 view.
+The IMAP protocol has a specific way to code folders names,
+especially when these names use non-ascii 7bit characters.
+This encoding is called utf7imap. 
 
+Imapsync uses the same encoding as IMAP, utf7imap.
+
+In order to well specify folders names on the command line, imapsync
+prints the complete folder list of both sides at the beginning of each run.
+
+The left column is the encoding you have to use, without the first enclosing 
+square brackets [], the right column is the human utf8 view.
+
+
+=======================================================================
+Q. What are the options to select folders?
+
+R. From https://imapsync.lamiral.info/README
+
+  OPTIONS/folders
+
+     --folder       str  : Sync this folder.
+     --folder       str  : and this one, etc.
+     --folderrec    str  : Sync this folder recursively.
+     --folderrec    str  : and this one, etc.
+
+     --folderfirst  str  : Sync this folder first. Ex. --folderfirst "INBOX"
+     --folderfirst  str  : then this one, etc.
+     --folderlast   str  : Sync this folder last. --folderlast "[Gmail]/All Mail"
+     --folderlast   str  : then this one, etc.
+
+     --nomixfolders      : Do not merge folders when host1 is case-sensitive
+                           while host2 is not (like Exchange). Only the first
+                           similar folder is synced. Example: with folders
+                           "Sent", "SENT" and "sent" on host1, only "Sent"
+                           will be synced to host2.
+
+     --skipemptyfolders  : Empty host1 folders are not created on host2.
+
+     --include      reg  : Sync folders matching this regular expression
+     --include      reg  : or this one, etc.
+                           If both --include --exclude options are used, then
+                           include is done before.
+     --exclude      reg  : Skips folders matching this regular expression
+                           Several folders to avoid:
+                            --exclude 'fold1|fold2|f3' skips fold1, fold2 and f3.
+     --exclude      reg  : or this one, etc.
+
+
+=======================================================================
+Q. Is it ok to use --subfolder1 and --subfolder2 to just select folders?
+
+R. No.
+
+  If you do:
+  
+    imapsync ... --subfolder1 Foo --subfolder2 Foo
+  
+  then it ends up with all messages from Foo going to Foo/INBOX
 
 
 =======================================================================
@@ -52,23 +130,15 @@ R. In the IMAP protocol each user can subscribe to one or more folders.
 
 
 =======================================================================
-Q. I want to exclude a folder hierarchy like "public"
+Q. How to exclude a folder hierarchy like "public"?
 
 R. Use:
 
---exclude "^public\."
-or maybe
---exclude '^"public\.'
-
-In the example given the character "." is the folder separator, you
-can omit it. Just take the string as it appears on the imapsync
-output line :
-
-From folders list : [INBOX] [public.dreams] [etc.]
+  --exclude "public"
 
 
 =======================================================================
-Q. I want to exclude only INBOX
+Q. How to exclude only the INBOX folder?
 
 R. Use:
 
@@ -79,8 +149,8 @@ A good way to see what will be done is to first use:
   imapsync ...  --exclude "^INBOX$" --justfolders --nofoldersizes --dry
 
 =======================================================================
-Q. I want to exclude folders matching SPAM no matter the case, 
-   aka how to be case insensitive
+Q. How to exclude folders matching SPAM no matter the case, 
+   aka, how to be case insensitive?
 
 R. Use:
 
@@ -92,7 +162,7 @@ A good way to see what will be done is to first use:
 
 
 =======================================================================
-Q. I want the --folder "MyFolder" option be recursive.
+Q. How to make option --folder "MyFolder" be recursive?
 
 Two solutions:
 
@@ -104,5 +174,11 @@ R2. Use --include "^MyFolder"
    Then the folder "MyFolder" and all its subfolders will be handled
    and only them.
 
+=======================================================================
+Q. How to change folders names or move them elsewhere?
+
+R. Read the document FAQ.Folders_Mapping.txt also available at
+   https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Mapping.txt
+
 =======================================================================
 =======================================================================

FAQ.d/FAQ.Folders_Sizes.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Folders_Sizes.txt,v 1.2 2018/05/24 11:34:30 gilles Exp gilles $
+$Id: FAQ.Folders_Sizes.txt,v 1.7 2022/04/23 13:45:15 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -9,18 +9,78 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Sizes.txt
   Imapsync tips to know folders sizes.
 =======================================================================
 
+Questions answered in this FAQ are:
 
+Q. How can I know the size of all folders before launching a sync?
+
+Q. After a sync, the folders sizes and the number of messages don't 
+   match between the source mailbox on host1 and the destination 
+   mailbox on host2.
+   Why? What can I do about it?
+
+Now the questions again with their answers.
 
 =======================================================================
 Q. How can I know the size of all folders before launching a sync?
 
 R. Use --justfoldersizes option.
 
-With --justfoldersizes option, imapsync displays the folder size
-part of a run and then exits without doing any transfer:
+With the --justfoldersizes option, imapsync evaluates and displays the 
+folders sizes on both sides and then exits without doing any transfer.
+It also presents the total size of each account, the number of messages,
+the biggest message size, the number of folders.
+
+Example:
 
   imapsync ... --justfoldersizes
 
+=======================================================================
+Q. After a sync, the folders sizes and the number of messages don't 
+   match between the source mailbox on host1 and the destination 
+   mailbox on host2.
+   Why? What can I do about it?
+
+R. There are several reasons to explain the different counts between
+   source and destination after a sync.
+   
+1) The sync ended prematurely, before syncing all wanted folders.
+   Solution: run new syncs until completion.
+
+2) There are messages on host2 that are not on host1, so host2 is
+   bigger than host1. Imapsync doesn't delete any message by default. 
+   Solution: use --delete2 to have a strict sync by folder, ie, 
+   delete messages on host2 that are not on host1.
+   
+3) There are folders on host2 that are not on host1, so host2 is 
+   bigger. Imapsync doesn't delete any folder by default.
+   Solution: use --delete2folders to have a strict sync, ie, 
+   delete folders on host2 that are not on host1.
+
+4) There are duplicate messages on host1. Imapsync doesn't sync 
+   duplicates by default. But it counts them and presents the number
+   found at the end on the line "Messages found duplicate on host1". 
+   To sync duplicates, use the option --syncduplicates
+
+5) There are cross-duplicate messages on host1, same messages in different
+   folders, and you use --gmail1 or --skipcrossduplicates, which avoid
+   copying cross-duplicate messages in more than one folder on host2.
+   Use --noskipcrossduplicates if you want imapsync to sync cross-duplicates.
+
+6) There are duplicate messages on host2 so host2 is
+   bigger than host1. Solution: use --delete2duplicates to delete
+   the host2 duplicates.
+
+7) Cosmic rays...
+
+The important part to take into account for a good and well done sync 
+is the last lines of imapsync. A good sync is when you encounter the
+three final lines:
+
+"The sync looks good, all xxxx identified messages in host1 are on host2."
+"There is no unidentified message on host1."
+"Detected 0 errors"
 
 =======================================================================
 =======================================================================
+
+ 

FAQ.d/FAQ.Fun_Things.txt

@@ -0,0 +1,80 @@
+#!/bin/cat
+$Id: FAQ.Fun_Things.txt,v 1.2 2021/10/15 13:37:05 gilles Exp gilles $
+
+This documentation is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Fun_Things.txt
+
+=======================================================================
+              Imapsync fun things to do with
+=======================================================================
+
+Questions answered in this FAQ are:
+
+Q. How can I have a daily digest of my mailbox?
+
+
+Now the questions again with their answers.
+
+=======================================================================
+Q. How can I have a daily digest of my mailbox?
+
+R. Use the same account as source and destination (not mandatory)
+
+  imapsync \
+         --host1 imap.example.com --user1 foo --password1 secret \
+         --host2 imap.example.com --user2 foo --password2 secret \
+         --folder INBOX --maxage 1 --dry --nodry1 --debugcontent \
+         --truncmess 1000 --f1f2 INBOX=Unused
+
+Some explanations on the options suggested:
+
+--folder INBOX to only focus on the folder INBOX
+
+--maxage 1 to select messages less old than 1 day
+
+--dry --nodry1 to avoid really syncing the messages
+
+--debugcontent to display the content
+
+--truncmess 1000 to display only the first 1000 bytes
+
+--f1f2 INBOX=Unused to force imapsync to do something. 
+  Since it is the same mailbox as source and destination, 
+  imapsync usually doesn't sync the messages as they are 
+  already at the destination. The folder Unused has to
+  exist, otherwise imapsync won't display the messages.
+  
+
+If you pipe this command with a filter like egrep, it becomes like
+magic:
+
+  imapsync \
+         --host1 imap.example.com --user1 foo --password1 secret \
+         --host2 imap.example.com --user2 foo --password2 secret \
+         --folder INBOX --maxage 1 --dry --nodry1 --debugcontent \
+         --truncmess 1000 --f1f2 INBOX=Unused \
+         | egrep 'From:|To:|Subject:|Date:|=====|msg '
+         
+Output example:
+
+$RCSfile: FAQ.Fun_Things.txt,v $ $Revision: 1.2 $ $Date: 2021/10/15 13:37:05 $ 
+msg INBOX/22359 copying to INBOX.Fake   (not really since --dry mode) ETA: vendredi 15 octobre 2021-10-15 15:31:38 +0200 CEST  57 s  38/38 msgs left
+================================================================================
+Subject: test: lei7eeCh4Fulahei
+To: <tata@localhost.info>
+Date: Mon, 11 Oct 2021 11:56:05 +0200 (CEST)
+From: gilles@localhost.info (Gilles LAMIRAL)
+================================================================================
+msg INBOX/22360 copying to INBOX.Fake   (not really since --dry mode) ETA: vendredi 15 octobre 2021-10-15 15:31:38 +0200 CEST  57 s  37/38 msgs left
+================================================================================
+Subject: test: yahShee3Ohvu9mai
+To: <tata@localhost.info>
+Date: Mon, 11 Oct 2021 11:56:05 +0200 (CEST)
+From: gilles@localhost.info (Gilles LAMIRAL)
+================================================================================
+
+Thanks to John Fedewa for this trick!
+
+=======================================================================
+=======================================================================

FAQ.d/FAQ.GDPR.txt

@@ -0,0 +1,65 @@
+$Id: FAQ.GDPR.txt,v 1.4 2019/09/12 10:07:27 gilles Exp gilles $
+
+This document is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.GDPR.txt
+
+=======================================================================
+ Imapsync and the GDPR, General Data Protection Regulation UE 2016/679
+=======================================================================
+
+Questions answered in this FAQ are:
+
+Q. Can you explain the compliance of imapsync with GDPR?
+   [General Data Protection Regulation UE 2016/679]
+
+
+Now the questions again with their answers.
+
+=======================================================================
+Q. Can you explain the compliance of imapsync with GDPR?
+   [General Data Protection Regulation UE 2016/679]
+
+R. 1) To know whether a newer imapsync exists or not, 
+      imapsync does a http GET to the file VERSION at
+      http://imapsync.lamiral.info/VERSION
+      
+ Via the HTTP User-agent header it also sends:
+  * the imapsync release
+  * the Perl version
+  * the Mail::IMAPClient version
+  * the Operating System
+  * the context ( Standard, CGI, or Docker )
+
+  You can remove this behavior by adding the option --noreleasecheck
+  on the command line or by setting $releasecheck = 0 in the source code. 
+  Check also https://nvd.nist.gov/vuln/detail/CVE-2013-4279
+
+  See also https://imapsync.lamiral.info/#NUMBERS
+
+2) Each imapsync run generates a logfile that corresponds exactly 
+   to the output printed on the console during the run. 
+   This log file contains data from the mailboxes, 
+   not the content messages, not the passwords but 
+   it contains the folders names and the login names.
+   Remove the log or use the option --nolog to avoid logging.
+
+3) In case you purchase imapsync:
+
+3.1) the data collected during the purchase,
+   Name, Company Name, Postal Address, VAT number, Price,
+   will be used to edit the invoice. I (Gilles LAMIRAL) keep a copy
+   of the invoice for accounting.
+   
+3.2)  Once in a while, I send an email to the buyers announcing 
+      a new imapsync is released and where to get it.
+   
+3.3) If the customer is a professional inside the EU, 
+   I also use his company VAT number and the amount to declare it
+   to the Customs each month, before the 10th of the month, at
+   https://pro.douane.gouv.fr/ 
+   It's mandatory, it's the law inside EU and the fine is 750 EUR/month
+   when it is not done or not well done a month.
+
+=======================================================================
+=======================================================================

FAQ.d/FAQ.General.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-# $Id: FAQ.General.txt,v 1.240 2018/11/26 11:49:56 gilles Exp gilles $
+# $Id: FAQ.General.txt,v 1.246 2022/06/08 10:57:52 gilles Exp gilles $
 
 =======================================================================
                      General FAQ for imapsync 
@@ -14,13 +14,15 @@ Questions answered in this FAQ are:
 Q. Do I need to create IMAP mailboxes at the destination platform?
 
 Q. Am I forced to publish the IMAP service on the Internet since the two
-   environment are not in the same location or same LAN?
+   environments are not in the same location or same LAN?
 
 Q. What are the most important differences between the Unix shell syntax
    and the Windows batch syntax.
 
 Q. How to install imapsync?
 
+Q. How to upgrade imapsync?
+
 Q. How to use imapsync?
 
 Q. Can you give some configuration examples?
@@ -29,11 +31,11 @@ Q. How can I have commercial support?
 
 Q. How can I have gratis support?
 
-Q. Where I can find old imapsync releases?
+Q. Where can I find old imapsync releases?
 
-Q. Where I can find free open and gratis imapsync releases?
+Q. Where can I find free open and gratis imapsync releases?
 
-Q. Is is legal to find imapsync gratis (or not) elsewhere?
+Q. Is it legal to find imapsync gratis (or not) elsewhere?
 
 Q. How "Facts and figures" are known 
    https://imapsync.lamiral.info/#NUMBERS
@@ -57,8 +59,8 @@ Q. How to verify imapsync.exe I got is the right file bit per bit?
 
 Q. Folders are not created on host2. What happens?
 
-Q. I am interested in creating  a local clone of the IMAP on a LAN 
-   server for  faster synchronizations, email will always be delivered 
+Q. I am interested in creating a local clone of the IMAP on a LAN 
+   server for faster synchronizations, email will always be delivered 
    to the remote server and so the synchronization will be one way - from 
    remote to local. How suited is imapsync for continuous one-way 
    synchronization of mailboxes? Is there a better solution?
@@ -77,17 +79,10 @@ Q. Is it possible to sync also the UIDs of the IMAP server?
 
 Q. The option --subscribe does not seem to work
 
-Q. With huge account (many messages) when it comes to reading the
-   destination server it comes out this error:
-   "To   Folder [INBOX.foobar] Not connected" 
-   What can I do?
-
 Q. Can Imapsync filter Spam during the sync?
 
 Q. How to migrate from uw-imap with an admin/authuser account?
 
-Q. How to migrate from cyrus with an admin account?
-
 Q: How to migrate from Sun Java Enterprise System / Sun One / iPlanet /
    Netscape servers with an admin account?
 
@@ -103,9 +98,6 @@ Q. I want to play with headers line and --regexmess but I want to leave
 Q. My imap server does not accept a message and warns
    "Invalid header". What is the problem?
 
-Q. The contact folder isn't well copied. 
-   How to copy the contact folder?
-
 Q: How can I write an .rpm with imapsync
 
 Q. Where I can read up on the various IMAP RFCs?
@@ -121,7 +113,7 @@ R. Yes!
 
 =======================================================================
 Q. Am I forced to publish the IMAP service on the Internet since the two
-   environment are not in the same location or same LAN?
+   environments are not in the same location or same LAN?
 
 R. The host where you run imapsync has to contact both servers via imap.
    You are not obliged to publish the imap service on the Internet if 
@@ -179,10 +171,16 @@ any \$1 by $1 (remove the \ before $).
 =======================================================================
 Q. How to install imapsync?
 
-R. Read the INSTALL files in the tarball. also available at
-   https://imapsync.lamiral.info/#doc
+R. Read the INSTALL files in the tarball. They are also available at
+   https://imapsync.lamiral.info/#install
+   and 
    https://imapsync.lamiral.info/INSTALL.d/
 
+=======================================================================
+Q. How to upgrade imapsync?
+
+R. Do like a fresh installation.
+
 =======================================================================
 Q. How to use imapsync?
 
@@ -222,16 +220,16 @@ I help all users as long as I have time to help them all,
 users who bought the support get my help first.
 
 =======================================================================
-Q. Where I can find old imapsync releases?
+Q. Where can I find old imapsync releases?
 
 R. Search the internet or ask the author.
 
 =======================================================================
-Q. Where I can find free open and gratis imapsync releases?
+Q. Where can I find free open and gratis imapsync releases?
 
 R. Search the internet or ask the author.
 
-Q. Is is legal to find imapsync gratis (or not) elsewhere?
+Q. Is it legal to find imapsync gratis (or not) elsewhere?
 
 R. Yes, the license permits it
    https://imapsync.lamiral.info/NOLIMIT
@@ -288,11 +286,11 @@ R. Add also --nofoldersizes since the default behavior is to compute
    useful to see what has to be done on each folder and guess when 
    the transfer will end (ETA).
 
-R. Add also --noexpungeaftereach if you use --delete. 
-   But be warn that an interrupted transfer can lose messages 
-   on host2 in a second run if you use a (silly) combination like
+R. Add also --noexpungeaftereach if you use --delete1. 
+   But be warned that an interrupted transfer can lose messages 
+   on host2 in a second run if you use a (silly) combination like:
 
-   imapsync ... --delete --noexpunge --noexpungeaftereach --expunge2
+   imapsync ... --delete1 --noexpunge1 --noexpungeaftereach --expunge2
 
 Notes about --useuid
 
@@ -374,13 +372,7 @@ Use --useuid then imapsync will avoid dealing with headers.
 =======================================================================
 Q. How can I try imapsync with latest Mail::IMAPClient 3.xx perl module?
 
-Three solutions at least.
-
-R1 - Look at the script named "i3" in the tarball, it can be used to
-     run imapsync with the included Mail-IMAPClient-3.39/ wherever you 
-     unpacked the imapsync tarball
-
-R2 Run:
+R1 Run:
 
    cpanm Mail::IMAPClient # this uses cpanminus
 
@@ -393,7 +385,7 @@ R2 Run:
    perl -MCPAN -e "install Mail::IMAPClient"
    
 
-R3 If you want to install the Perl module locally in a directory
+R2 If you want to install the Perl module locally in a directory
 
    - Download latest Mail::IMAPClient 3.xx at
      http://search.cpan.org/dist/Mail-IMAPClient/
@@ -411,11 +403,11 @@ R3 If you want to install the Perl module locally in a directory
      or if imapsync is in directory /path/
         perl -I./Mail-IMAPClient-3.39/lib   /path/imapsync ...
 
-
 =======================================================================
 Q. How can I use imapsync with Mail::IMAPClient 2.2.9 perl module?
 
 R. Mail::IMAPClient 2.2.9 is no longer supported.
+   Help yourself.
 
 =======================================================================
 Q. How to verify imapsync.exe I got is the right file bit per bit?
@@ -512,8 +504,8 @@ R. imapsync doesn't talk POP3 but I think you mean UID in IMAP.
 Q. Is it possible to sync also the UIDs of the IMAP server?
 
 R. UIDs in IMAP are chosen and created by the server, not by the 
-   client software. imapsync is a client software. So UIDs cannot 
-   be synced by any imap method.
+   client software. imapsync is a client software command. 
+   So UIDs cannot be synced by any imap method.
 
    UIDs may be synced via a rsync command between the imap servers but
    it implies they are the same software, among other constraints.
@@ -526,16 +518,6 @@ R1. Use it with --subscribed
 R2. There is also the --subscribe_all option that subscribe
     to all folders on host2.
 
-=======================================================================
-Q. With huge account (many messages) when it comes to reading the
-   destination server it comes out this error:
-   "To   Folder [INBOX.foobar] Not connected" 
-   What can I do?
-
-R. May be spending too much time on the source server, the connection 
-   timed out on the destination server.
-   Try options --nofoldersizes 
-
 ======================================================================
 Q. Can Imapsync filter Spam during the sync?
 
@@ -560,46 +542,6 @@ R. Use the following syntax:
   imapsync ... --user1="loginuser*admin_user" --password1 "admin_user_password"
 
 
-======================================================================
-Q. How to migrate from cyrus with an admin account?
-
-R. Use:
- 
- imapsync ... \
-   --authuser1 admin_user  ----password1 admin_user_password \
-   --user1 foo_user --ssl1
-
-Instead of --ssl1 the alternative --tls1 can be used.
-With --authuser1, the option --authmech1 PLAIN is set
-automatically, you don't have to add it.
-
-PLAIN authentication is the only way to go with --authuser1 for now.
-So don't use --authmech1 SOMETHING with --authuser1 admin_user,
-it will not work.
-Same behavior with the --authuser2 option.
-
-Do not forget the option --ssl1 or --tls1 since PLAIN auth is only
-supported with ssl encryption most of the time. But it can
-work without --ssl1 nor --tls1 if PLAIN is permitted in clear text
-transmissions (the normal mode).
-
-Add the AdminAccount to admins line in /etc/imapd.conf
-Give AdminAccount lrswipkxtecda to the Cyrus Imap account 
-being migrated from, "joe" here.
-
-
-Here is an example:
- imapsync \
-	--host1 server1 \
-	--user1 joe \
-	--authuser1 AdminAccount \
-	--password1 AdminAccountPassword \
-	--ssl1 \
-	--host2 server2 \
-	--user2 joe \
-	--password2 joespassonserver2 \
-	--exclude "^user\."
-
 ======================================================================
 Q: How to migrate from Sun Java Enterprise System / Sun One / iPlanet /
    Netscape servers with an admin account?
@@ -628,20 +570,6 @@ In imapsync, you can achieve this by using the following options:
   --proxyauth1 \
   --passfile admin.txt
 
-======================================================================
-Q. Is there a way to delete the destination folder when the source 
-   folder is no longer there?
-
-R. Yes, use --delete2folders
-
---delete2folders       : Delete folders in host2 that are not in host1 server. 
-                         For safety, first try it like this (it is safe):
-			 --delete2folders --dry --justfolders --nofoldersizes
---delete2foldersonly   <regex>: Deleted only folders matching regex.
-                         Example: --delete2foldersonly "/^Junk$|^INBOX.Junk$/"
---delete2foldersbutnot <regex>: Do not delete folders matching regex.
-                         Example: --delete2foldersbutnot "/Tasks$|Contacts$|Foo$/"
-
 =======================================================================
 Q. I would love to have a function to inject lines in the header. 
    Things like "X-migrated-from-foo: 20100617"
@@ -716,23 +644,10 @@ Solution e) is solution a) made by imapsync itself.
 Solutions c) and d) keep "From " lines information
 (normally it's useless to keep them)
 
-Best solutions are e) or d).
-
-=======================================================================
-Q. The contact folder isn't well copied. 
-   How to copy the contact folder?
-
-R. Forget the destination server (choose the same)
-Change the script around line 1426
-       # ITSD
-       $new_id = $from->copy($t_fold,$f_msg);
-       #$new_id = $to->append_string($t_fold,$string, $flags_f, $d);
-
-and tried a copy of the mail instead an append_string. Because we are
-using the same server, we can use $from->copy Therefore we seem to not
-download and upload the message and therefore we do not have any
-format issues. And now it works fine. (Thanks to Hansjoerg.Maurer)
-
+Best solutions are e) or d). 
+I prefer the d) solution because it fixes the issue
+and keeps the old "From" value while the e) solution removes 
+the old "From" value definitively.
 
 ======================================================================
 Q: How can I write an .rpm with imapsync

FAQ.d/FAQ.Gmail.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Gmail.txt,v 1.51 2019/02/18 09:49:14 gilles Exp gilles $
+$Id: FAQ.Gmail.txt,v 1.92 2022/06/28 11:03:40 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -11,6 +11,7 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt
 
 Questions answered in this FAQ are:
 
+
 Q. Can I use imapsync to transfer from or to Gmail accounts?
 
 Q. Even after several syncs the number of messages differs
@@ -19,6 +20,10 @@ Q. Even after several syncs the number of messages differs
 
 Q. How many days does it take to transfer X GB?
 
+Q. What is the Gmail free maximum storage size? 
+R. 15 GB in 2020. Need more?
+   https://gsuite.google.com/pricing.html
+
 Q. How to synchronize from Gmail to Gmail?
 
 Q. How to synchronize from XXX to Gmail?
@@ -26,14 +31,34 @@ Q. How to synchronize from XXX to Gmail?
 Q. How to synchronize from Gmail to XXX?
 
 Q. I can't authenticate with Gmail via IMAP
-   and Gmail says "Please log in via your web browser"
+   and Gmail says 
+   "Please log in via your web browser"
+   or it says 
+   "Application-specific password required: https://support.google.com/accounts/answer/185833 (Failure)"
 
 Q. Can not open imap connection on [imap.gmail.com]
    Unable to connect to imap.gmail.com
 
-Q. Can I transfer "Chat" messages with imapsync?
+Q. I want to sync the Sent folder only, what is it on Gmail?
 
-Q. Can I safely use --useuid for Gmail transfers?
+Q. From Gmail to Gmail, I want to keep Categories. What can I do?
+
+Q. The context is a Gmail account at the source, but not Gmail at the
+destination. I want cross-duplicates, different Gmail folders/labels
+generating duplicate messages across folders on the destination,
+except for the "All Mail" folder.
+
+Q. I can't sync more than 1000 messages in each Gmail folder.
+   What can I do?
+
+Q. What are the Gmail imap names of the folders 
+   "Sent" "Trash" "Drafts" "Junk" "Flagged" and "All"?
+
+Q. Can I transfer Gmail "Chat" messages with imapsync?
+
+Q. Can I safely use --useuid or --usecache for Gmail transfers?
+
+Q. Can I use --syncduplicates Gmail transfers?
 
 Q. Gmail does not really delete messages in folder [Gmail]/All Mail
    What happens? What can I do?
@@ -62,10 +87,50 @@ Q. Can I use imapsync to transfer from or to Gmail accounts?
 
 R. Yes. But IMAP access to a Gmail account is not allowed by default so 
 it has to be allowed in the Gmail configuration part:
- -> Settings
- -> Forwarding and POP/IMAP
- -> IMAP Access
- -> Enable IMAP
+ -> Do not go to "Manage your Google Account" (the upper right icon with your Google face)
+ -> Instead, go to Gmail at https://mail.google.com/
+ -> Click on Settings, the gear button on the upper right of Gmail.
+ -> Click on "See all settings" at the top of the pull down menu.
+ -> Go to the tab "Forwarding and POP/IMAP"
+ -> Go down to "IMAP Access"
+ -> Activate the radio button "Enable IMAP"
+ -> Click on the button "Save changes" at the bottom of the page.
+
+That's not finished, there is also the authentication issue to consider.
+There are two different ways to authenticate to Gmail with Imapsync,
+either:
+
+1) "App password with 2-step verification"
+
+or
+
+2) XOauth2 ready for groups on any OS or 
+   for individual Gmail authentication on Linux.
+
+For the second one, XOauth2, see the document 
+https://imapsync.lamiral.info/FAQ.d/FAQ.XOAUTH2.txt
+
+
+1) "App password with 2-step verification" is the first way to 
+   authenticate to Gmail with Imapsync.
+
+Go to
+https://myaccount.google.com/security
+and sign in with the account you want to transfer email to or from.
+Go to the "Signing in to Google" part of the security page and follow
+the process to activate the "2-step verification" until it is ok.
+
+Then, just below "2-step verification" click on the "App passwords"
+and follow the instruction to get a 16 characters password, on the
+pull down button "Select app", choose "Other (Custom name)" and call
+it imapsync for example.  Now use this this password to access the
+mailbox with imapsync.
+
+The following document details what I've just described:
+https://support.google.com/accounts/answer/185833?hl=en
+Even if Google says "Every App Password is only used once",
+it's false and the password can be used many times.
+
 
 =======================================================================
 Q. Even after several syncs the number of messages differs
@@ -83,20 +148,27 @@ multiple messages and it doesn't match what imapsync presents.
 Q. How many days does it take to transfer X GB?
 
 R. Basically it takes X days to transfer X GB per account.
-   Gmail has usage limits per day and use throttlers when
+   Gmail has usage limits per day and use throttles when
    they are overtaken
-   http://support.google.com/a/bin/answer.py?hl=en&answer=1071518
+   https://support.google.com/a/answer/1071518?hl=en
    From the previous link:
 
-   * it's 2X days to upload  X GB to Gmail, it's why I suggest to add
-   --maxbytespersecond 10000
+   * Upload with IMAP: 500 MB per day.
+   It's 2X days to upload  X GB to Gmail, it's why I suggest to add
+   --maxbytespersecond 20000 --maxbytesafter 1_000_000_000
    for uploading messages to Gmail
-   * it's X/2 days to download X BG from Gmail, it's why I suggest to add
-   --maxbytespersecond 20000
+   (should be --maxbytespersecond 5787 --maxbytesafter 500_000_000
+   but the higher values given work well)
+   
+   * Download with IMAP: 2500 MB per day.
+   It's X/2.5 days to download X GB from Gmail, it's why I suggest to add
+   --maxbytespersecond 40_000 --maxbytesafter 3_000_000_000
    for downloading messages from Gmail
+   (should be --maxbytespersecond 28_935 --maxbytesafter 2_500_000_000
   
-   That's theoretical values that always work in practice. Try
-   upper values and see if they still work.
+   That's empirical values that work in practice.
+   Try upper values and see if they still work. 
+   I would be grateful if you report them to me.
 
    How Gmail says limits are reached?
    This is either a disconnection with
@@ -116,25 +188,31 @@ imapsync  \
            --password2 gmailsecret2 \
            --gmail1 --gmail2 
 
-Or, replacing what does --gmail1 --gmail2 (values are from imapsync 1.882):
+Or, replacing what does --gmail1 --gmail2 (values are from imapsync 2.178):
 
 imapsync \
            --host1 imap.gmail.com \
+           --ssl1 \
            --user1 account1@gmail.com \
            --password1 gmailsecret1 \
            --host2 imap.gmail.com \
+           --ssl2 \
            --user2 account2@gmail.com \
            --password2 gmailsecret2 \
            --maxbytespersecond 20_000 \
            --maxbytesafter 1_000_000_000 \
-           --maxsleep 2 \
            --automap \
-           --exclude "\[Gmail\]$"   
+           --maxsleep 2 \
+           --useheader X-Gmail-Received --useheader Message-Id \
+           --synclabels \
+           --resynclabels \
+           --exclude "\[Gmail\]$" \
+           --folderlast  "[Gmail]/All Mail"
 
 With --gmail1 --gmail2 you can override the parameters
 that are activated by them, for example:
 
-imapsync ... --gmail1 --gmail2 --maxbytespersecond 50_000
+imapsync ... --gmail1 --gmail2 --maxbytespersecond 50_000 --noautomap
 
 Explanations:
 
@@ -189,29 +267,40 @@ R. Use the following example:
 
 imapsync --host1 mail.oldhost.com     \
          --user1 my_email@oldhost.com \
-         --password1 password   \
+         --password1 passwordold   \
          --user2 my_email@gmail.com \
-         --password2 password \
-         --gmail2 --maxsize 35_651_584
+         --password2 gmailsecret \
+         --gmail2
 
-Or, replacing what does --gmail2 (values are from imapsync 1.882):
+Or, replacing what does --gmail2 (values are from imapsync 2.178):
+(For Linux only because of the quotes)
 
 imapsync --host1 mail.oldhost.com     \
          --user1 my_email@oldhost.com \
-         --password1 password   \
+         --password1 passwordold   \
          --host2 imap.gmail.com \
+         --ssl2 \
          --user2 my_email@gmail.com \
-         --password2 password \
+         --password2 gmailsecret \
          --maxbytespersecond 20_000 \
          --maxbytesafter 1_000_000_000 \
+         --automap    \
+         --expunge1   \
+         --addheader  \
          --maxsleep 2 \
-         --maxsize 35_651_584 \
-         --automap   \
-         --expunge1  \
-         --addheader \
-         --exclude "\[Gmail\]$"      \
-         --regextrans2 "s/[ ]+/_/g"  \
-         --regextrans2 "s/['\^\"\\\\]/_/g" 
+         --maxsize 35_651_584
+         --exclude "\[Gmail\]$"  \
+         --regextrans2 's,\[Gmail\].,,' \
+         --regextrans2 's,^ +| +$,,g'
+         --regextrans2 's,/ +| +/,/,g'
+         --regextrans2 's/['"'"'\\^"]/_/g'
+         --folderlast  "[Gmail]/All Mail"
+
+
+Starting from imapsync 1.926 a --regextrans2 replacing 
+all blanks, was:
+
+        --regextrans2 "s/[ ]+/_/g" 
 
 With --gmail2 you can override the parameters
 that are activated by them, for example:
@@ -235,12 +324,11 @@ be 500_000_000, not 1_000_000_000, but 1GB is usually ok.
 --maxsleep 2 is to avoid sleeping more than 2 seconds when 
 the --maxbytespersecond value implies a long sleep between 2 copies.
 
---maxsize 25_000_000 is mandatory since Gmail limits messages size
-up to 25 MB. This value increases over time, it was 10 MB some 
-years ago so you can try higher values. The Gmail page about
-this limit is https://support.google.com/mail/answer/6584
+--maxsleep 0 cancels any imapsync bandwidth restriction given by
+the options --maxbytespersecond or --maxmessagespersecond
+It does not cancel the imap server bandwidth restrictions.
 
---automap is optional but it will save manual folder names
+--automap is optional but it will save manual folders names
 changes or the use of --regextrans2 or --f1f2 to map folder names.
 
 --expunge1 is optional. It deletes messages marked \Deleted on host1.
@@ -277,7 +365,7 @@ On Windows:
 --regextrans2 "s/[\^]/_/g" is mandatory.  It converts, since
 not accepted by gmail, character ^ to character _ underscore.
 
---regextrans2 "s/['\"\\\\]/_/g" is optional. It converts 
+--regextrans2 's/['"'"'\\^"]/_/g' is optional. It converts 
 characters ' or " or \ to character _ underscore.
 
 You can select folders exported to imap within the gmail preferences.
@@ -291,27 +379,31 @@ Q. How to synchronize from Gmail to XXX?
 R. Use this example:
 
 imapsync \
-           --user1 gilles.lamiral@gmail.com \
+           --user1     foo@gmail.com \
            --password1 gmailsecret \
-           --host2 localhost \
-           --user2 tata \
+           --host2     imap.example.org \
+           --user2     tata \
+           --password2 tatasecret \
            --gmail1
 
-Or, replacing what does --gmail1 (values are from imapsync 1.882):
+Or, replacing what does --gmail1 (values are from imapsync 2.178):
 
 imapsync \
-           --host1 imap.gmail.com \
-           --user1 gilles.lamiral@gmail.com \
+           --host1     imap.gmail.com \
+           --ssl1 \
+           --user1     foo@gmail.com \
            --password1 gmailsecret \
-           --host2 localhost \
-           --user2 tata \
+           --host2     imap.example.org \
+           --user2     tata \
            --password2 tatasecret \
-           --maxbytespersecond 20_000 \
+           --maxbytespersecond 40_000 \
+           --maxbytesafter 3_000_000_000 \
+           --automap \
+           --maxsleep 2 \
+           --skipcrossduplicates \
            --useheader="X-Gmail-Received" \
            --useheader "Message-Id" \
-           --automap \
            --regextrans2 "s,\[Gmail\].,," \
-           --skipcrossduplicates \
            --folderlast  "[Gmail]/All Mail"
 
 With --gmail1 you can override the parameters
@@ -321,56 +413,59 @@ imapsync ... --gmail1 --maxbytespersecond 50_000 --noskipcrossduplicates --noaut
 
 Explanations:
 
---maxbytespersecond 20_000 ( 20 kBytes/s ) option is here to avoid 
+--maxbytespersecond 40_000 ( 40 kBytes/s ) option is here to avoid 
 locking or errors when imap transfers exceed Gmail maximum limit. 
 See http://support.google.com/a/bin/answer.py?hl=en&answer=1071518
---maxbytespersecond 20_000 is not mandatory in the sense Gmail may 
-allow you to use an upper value than 2 GBytes per 24h without 
+--maxbytespersecond 40_000 is not mandatory in the sense Gmail may
+allow you to use an upper value than 2 GBytes per 24h without
 disconnections.
 
 --useheader="X-Gmail-Received" --useheader "Message-Id" are not
-mandatory. I use them because I found (several years ago, it may
-have changed) that Gmail always adds a different header 
-"X-Gmail-Received:" to all messages it gets. So the identification
-by imapsync can not fail using this header. "Message-Id" is there
-for safety about this Gmail rule.
+mandatory. I use them because I found (several years ago, it may have
+changed) that Gmail always adds a different header "X-Gmail-Received:"
+to all messages it gets. So the identification by imapsync can not
+fail using this header. "Message-Id" is there for safety about this
+Gmail rule.
 
---automap is optional but it will save manual folder names
-changes or the use of --regextrans2 to map folder names.
+--automap is optional but it will save manual folder names changes or
+the use of --regextrans2 to map folder names.
 
 --regextrans2 "s,\[Gmail\].,,"
-If your destination imap server doesn't like "[Gmail]" name, 
+If your destination imap server doesn't like the "[Gmail]" name, 
 get rid of this "[Gmail]" part with that.
 
-You can select folders exported to imap within the gmail preferences,
-for example you may unselect all "System labels".
+You can select folders exported to imap within the Gmail preferences,
+for example, you may unselect all "System labels".
 
 --skipcrossduplicates is optional but it can save Gigabytes of hard
-disk memory. Within imap protocol, Gmail presents Gmail labels as 
-folders, so a message labeled "Work" "ProjectX" "Urgent" ends up 
-in three different imap folders "Work" "ProjectX" and "Urgent"
-after an imap sync. Option --skipcrossduplicates prevent this behavior.
+disk memory. Within imap protocol, Gmail presents Gmail labels as
+folders, so a message labelled "Work" "ProjectX" "Urgent" ends up in
+three different imap folders "Work" "ProjectX" and "Urgent" after an
+imap sync. Option --skipcrossduplicates prevent this behavior.
 
-An issue with --skipcrossduplicates is that the first label synced
-by imapsync goes to its corresponding folder but other labels are
-then ignored. You can choose what labels have the 
-priority by using the --folderfirst option. For example
---folderfirst "Work" will sync messages labeled "Work" before
-messages labeled "CanWait" or "Urgent". By default imapsync 
-syncs folders (Gmail labels) using the classical alphanumeric order.
+An issue with --skipcrossduplicates is that the first label synced by
+imapsync goes to its corresponding folder but other labels are then
+ignored. You can choose what labels have priority by using the
+--folderfirst option. For example, --folderfirst "Work" will sync
+messages labelled "Work" before messages labelled "CanWait" or
+"Urgent". By default imapsync syncs folders (Gmail labels) using the
+classical alphanumeric order.
 
---folderlast "CanWait" will sync only messages that have the
-label CanWait and only it.
+Option --folderlast "CanWait" will sync only messages that only have
+the label CanWait.
 
---folderlast "[Gmail]/All Mail", in conjunction with option 
---skipcrossduplicates, will only put in "[Gmail]/All Mail"
-the messages that are not labeled at all.
+Option --folderlast "[Gmail]/All Mail", in conjunction with option
+--skipcrossduplicates will only put in "[Gmail]/All Mail" the messages
+that are not labelled at all.
 
 
 https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt
 =======================================================================
 Q. I can't authenticate with Gmail via IMAP
-   and Gmail says "Please log in via your web browser"
+   and Gmail says 
+   "Please log in via your web browser"
+   or it says 
+   "Application-specific password required: https://support.google.com/accounts/answer/185833 (Failure)"
 
 R0. My current settings on two Gmail accounts are made
 with three steps 0) 1) 2)
@@ -386,6 +481,7 @@ https://support.google.com/accounts/answer/6010255?hl=en
 
 R1. See Coert Grobbelaar solution:
 https://web.archive.org/web/20150906230041/http://security.stackexchange.com/questions/86404/how-do-i-interact-with-google-to-import-email-via-imapsync
+https://support.google.com/accounts/answer/185833
 
 R2. I had the same issue one time (mars 2015) logging to Gmail 
 with imapsync.
@@ -395,6 +491,48 @@ it asked me to receive a code via a mobile, I said yes,
 I entered the code and everything went ok on next syncs
 with imapsync. 
 
+R1 again. Since people rarely follow links in a txt file, I reproduce
+Google documentation from the link
+https://support.google.com/accounts/answer/185833
+
+Sign in using App Passwords
+An App Password is a 16-digit passcode that gives a non-Google app or 
+device permission to access your Google Account. 
+App Passwords can only be used with accounts that 
+have 2-Step Verification turned on.
+
+When to use App Passwords
+To help keep your account secure, use "Sign in with Google" to 
+connect apps to your Google Account. 
+If the app you’re using doesn’t offer this option, you can either:
+
+Use App Passwords to connect to your Google Account
+Switch to a more secure app
+Create & use App Passwords
+Note: If you use 2-Step-Verification 
+( https://support.google.com/accounts/answer/185839 )
+and are seeing a "password incorrect" error when trying to access 
+your Google Account, an App Password may solve the problem.
+
+* Go to your Google Account https://myaccount.google.com/
+* On the left navigation panel, choose Security.
+* On the "Signing in to Google" panel, choose App Passwords. 
+  If you don’t see this option:
+  * 2-Step Verification is not set up for your account.
+  * 2-Step Verification is set up for security keys only.
+  * Your account is through work, school, or other organization.
+  * You’ve turned on Advanced Protection for your account.
+* At the bottom, choose Select app and choose the app you’re using.
+* Choose Select device and choose the device you’re using.
+* Choose Generate.
+* Follow the instructions to enter the App Password. 
+  The App Password is the 16-character code in the yellow bar on your device.
+* Choose Done.
+
+Most of the time, you’ll only have to enter an App Password once per 
+app or device, so don’t worry about memorizing it.
+
+
 
 =======================================================================
 Q. Can not open imap connection on [imap.gmail.com]: 
@@ -428,8 +566,160 @@ Or go to http://ping.eu/nslookup/ to get the resolution.
 
 Thanks to Chris Nolan to report, understand and fix this issue!
 
+
+
 =======================================================================
-Q. Can I transfer "Chat" messages with imapsync?
+Q. I want to sync the Sent folder only, what is it on Gmail?
+
+R1. Use:
+
+  imapsync ... --folder "[Gmail]/Sent Mail"
+
+R2. In fact, by configuration from the user, it can be another folder 
+    name presented in IMAP. Use --automap --dry to see this mapping.
+    See below 
+
+=======================================================================
+Q. From Gmail to Gmail, I want to keep Categories. What can I do?
+
+R. First, read a document to learn how to play with Gmail Categories:
+https://support.google.com/mail/answer/3094499
+
+Categories are not presented in IMAP in Gmail, so synchronizing them 
+with imapsync requires some work.
+
+Let's say I want to synchronize the category named "Updates".
+
+1. Create a label "Updates_L", or name it as you want. You can't 
+   create a label named "Updates" like the category name because
+   Gmail will refuse to create it, saying it's a reserved name.
+
+2. Select all messages in the category "Updates".
+   To do so, you can use the string "category:updates" in the 
+   search input text located at the top of the Gmail page.
+   Notice the case is irrelevant to the search,
+   "category:updates" equals "Category:Updates".
+
+   Gmail then announces "All 100 messages on this page are selected."
+   It's not enough, I want all the "Updates" category messages.
+   So, on the right of this statement, click on the text 
+   "Select all messages that match this search".
+   Now Gmail announces "All messages in this search are selected."
+
+3. Apply label "Updates_L" created at step 1 to all the selected
+   messages. Gmail will present a warning and a 
+   "Confirm bulk action" button. Press "Ok".
+
+4. Repeat steps 1 to 3 for each category you want to synchronize.
+
+5. Transfer your mailbox to the new account with imapsync using the
+   options --gmail1 --gmail2, the labels will be synchronized.
+   
+6. For each of the labels you made for categories, do the steps 7 to 9.
+   As an example, I take the label "Updates_L" previously used.
+
+7. Select all messages with the label "Updates_L".
+   You can use the string "label:Updates_L" in the search input 
+   text located at the top of the Gmail page.
+   Gmail then announces "All 100 messages on this page are selected."
+   It's not enough, I want all the "updates_l" labeled messages.
+   So, on the right of this statement, click on the text 
+   "Select all messages that match this search".
+   Now Gmail announces "All messages in this search are selected."
+   (does it sound similar to a previous §?)
+   
+8. Apply the category "Updates" to all the selected
+   messages. Gmail will present a warning and a 
+   "Confirm bulk action" button. Press "Ok".
+   If you search where are the categories: the categories are 
+   located with the labels, as if they were labels.
+
+9. Delete the temporary label. That part is optional.
+
+Maybe one day Google will make categories optionally available
+in IMAP, like any other label and this trick will be obsolete.
+Labels can be "Show in IMAP" at
+https://mail.google.com/mail/u/0/#settings/labels
+Categories are listed there but not possibly "Show in IMAP", not yet :-)
+
+Thanks to Michael Leggett https://twitter.com/leggett for this 
+trick. See the story at
+https://twitter.com/leggett/status/1484632316697739267?s=20
+
+=======================================================================
+Q. The context is a Gmail account at the source, but not Gmail at the
+destination. I want cross-duplicates, different Gmail folders/labels
+generating duplicate messages across folders on the destination,
+except for the "All Mail" folder.
+
+R. This behavior, different Gmail labels generating duplicates across
+folders except for the "All Mail" folder, is obtained with a
+synchronization with no --gmail1 but instead --exclude "All Mail"
+followed by another synchronization with --gmail1 but without
+--exclude "All Mail".
+
+  imapsync ... --exclude "All Mail"
+  imapsync ... --gmail1
+
+This second sync is done to get the messages with no labels at all
+from the folder "All Mail", if they are some of them.
+
+Technically speaking, the first synchronization has not the option
+--skipcrossduplicates but the second one has the option
+--skipcrossduplicates and it also performs the Gmail "All Mail"
+folder/label at the end so only the messages that have no labels will
+end up in the "All Mail" destination folder (which is not Gmail,
+remember the context).
+
+Thanks to Mike Alexander for this suggestion. 
+Mike also provided a patch to implement that directly in imapsync.
+See https://github.com/imapsync/imapsync/pull/266
+(I refused it because well maintaining it will be very hard for me
+and I have other priorities).
+
+=======================================================================
+Q. I can't sync more than 1000 messages in each Gmail folder.
+   What can I do?
+
+R. It's a Gmail configuration feature that someone set for your
+Gmail account. Go to 
+https://mail.google.com/mail/u/0/?tab=wm#settings/fwdandpop
+and select 
+"Do not limit the number of messages in an IMAP folder (default)"
+
+=======================================================================
+Q. What are the Gmail imap names of the folders 
+   "Sent" "Trash" "Drafts" "Junk" "Flagged" and "All"?
+
+R. By default, Gmail imap names of the folders 
+   "Sent" "Trash" "Drafts" "Junk" "Flagged" and "All"
+   are like this:
+
+Turned on automapping folders ( use --noautomap to turn off automapping )
+Host1: special [Gmail]/All Mail     = \All
+Host1: special [Gmail]/Drafts       = \Drafts
+Host1: special [Gmail]/Sent Mail    = \Sent
+Host1: special [Gmail]/Spam         = \Junk
+Host1: special [Gmail]/Starred      = \Flagged
+Host1: special [Gmail]/Trash        = \Trash
+
+Host2: special [Gmail]/All Mail     = \All
+Host2: special [Gmail]/Bin          = \Trash
+Host2: special [Gmail]/Drafts       = \Drafts
+Host2: special [Gmail]/Sent Mail    = \Sent
+Host2: special [Gmail]/Spam         = \Junk
+Host2: special [Gmail]/Starred      = \Flagged
+
+This is an excerpt from the imapsync output log done with the 
+--automap option. This listing is located just before the 
+folders listings.
+
+This mapping can be changed by configuration in the Gmail 
+web interface so the result may vary.
+
+
+=======================================================================
+Q. Can I transfer Gmail "Chat" messages with imapsync?
 
 R. No. 
    Even if you think you can because the Chat folder is seen in IMAP,
@@ -438,9 +728,10 @@ R. No.
    using the IMAP protocol.
 
 =======================================================================
-Q. Can I safely use --useuid for Gmail transfers?
+Q. Can I safely use --useuid or --usecache for Gmail transfers?
 
-R. Yes, but I suggest to not use --useuid for Gmail transfers.
+R. Yes, but I suggest to not use --useuid nor --usecache 
+   for Gmail transfers.
 
 Using UIDs is useless with Gmail in the case of global duplicates
 (duplicates across different folders). Gmail always accept a global
@@ -448,33 +739,44 @@ duplicate message as a new message, giving imapsync a new UID for this
 message, and throw it away because it already has it. Gmail
 will do this at each run so imapsync will always try to copy the
 message, and Gmail will always accept and throw away the new copy. It
-ends up with no duplicates on Gmail but a waste of bandwith and time,
+ends up with no duplicates on Gmail but a waste of bandwidth and time,
 which is the opposite goal of --usecache implied by --useuid.
 
+=======================================================================
+Q. Can I use --syncduplicates Gmail transfers?
+
+R. Yes, but Gmail will accept the new duplicate message and 
+throw away the new copy. It ends up with no duplicates on Gmail but 
+a waste of bandwidth and time.
 
 ======================================================================= 
-Q. Gmail does not really delete messages in folder [Gmail]/All Mail 
-   What happens? What can I do? 
+Q. Gmail does not really delete messages in folder [Gmail]/All Mail
+   What happens? What can I do?
 
 R. It's true and explained in Gmail documentation at
-https://support.google.com/mail/answer/7401?hl=en
+https://support.google.com/mail/answer/7401?hl=en#zippy=%2Cempty-your-trash
 
 To really remove messages in folder "[Gmail]/All Mail", they have to
 be moved to the "Trash" folder and be deleted from "Trash".
 
-Quoting the link above (25 december 2018, happy christmas by the way!)
+Quoting the link above (4 february 2022)
 ...
 Empty your Trash
-If you don't want a message to stay in your Trash for 30 days, 
-you can permanently delete it.
-* On your computer, go to Gmail.
-* On the left side of the page, scroll down, 
-  then click More and then Trash.
-* Check the box next to messages you want to permanently delete, 
-  then click Delete forever.
 
-* Note: The Delete forever option is only available in the Trash. 
-  You won’t be able to delete items forever from the Search result view.
+If you don't want a message to stay in your trash for 30 days,
+you can permanently delete it.
+
+* On your computer, go to Gmail.
+
+* On the left side of the page, click More.
+
+* Scroll to Trash.
+
+* Check the box next to messages you want to permanently delete.
+  Then click Delete forever.
+  
+* Tip: The "Delete forever" option is only available in the trash.
+  You won’t be able to delete items permanently from the Search result view.
   To delete all messages in your trash, click Empty Trash now.
 
 
@@ -546,7 +848,6 @@ xxx sub-folders show up nested within xxx/xxx.
 Q. How to migrate email from gmail to google apps?
 
 R. Take a look at:
-http://www.linux-france.org/prj/imapsync_list/msg00639.html
 
 http://biasecurities.com/2009/02/migrate-email-from-gmail-to-google-apps/
 http://www.thamtech.com/blog/2008/03/29/gmail-to-google-apps-email-migration/

FAQ.d/FAQ.GoDaddy.txt

@@ -0,0 +1,23 @@
+$Id: FAQ.GoDaddy.txt,v 1.1 2021/05/21 18:19:29 gilles Exp gilles $
+
+This documentation is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.GoDaddy.txt
+
+
+=======================================================================
+             Imapsync tips for GoDaddy
+=======================================================================
+
+
+Questions answered in this FAQ are:
+
+Q. What are the GoDaddy settings for imapsync?
+
+Now the questions again with their answers.
+
+=======================================================================
+Q. What are the GoDaddy settings for imapsync?
+
+R. The imap server is imap.secureserver.net
+

FAQ.d/FAQ.ISP.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.ISP.txt,v 1.9 2018/05/24 11:34:30 gilles Exp gilles $
+$Id: FAQ.ISP.txt,v 1.14 2022/05/22 08:04:05 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -9,36 +9,26 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.ISP.txt
       Imapsync tips for ISP. Specific issues and solutions.
 =======================================================================
 
-* IMAPSync - usage scenario with ISP - by Flávio Zarur Lucarelli.
+* IMAPSync - usage scenario with ISP - by Flavio Zarur Lucarelli
+  (https://www.lucanet.com.br)
 
-I thought Id write a quick step by step on my attempts to learn the
-imapsync features that matter the most, so it works as we expected in
-the cenario in which we use it, which is to migrate customers from
+I thought I had to write a quick step by step on my attempts to learn
+the imapsync features that matter the most, so it works as we expected
+in the scenario in which we use it, which is to migrate customers from
 their old ISP to our ISP/email hosting. Thanks to the master Gilles
 Lamiral for all his help and hard work.
 
 First of all, remember to use --dry to test things first always and
 check the log file to see what would actually happen.
 
-Initially, I used a method where I'd do an exact sync of source to
-destination, deleting messages which were in destination, but not
-source. I call this "Method 2", below. In this cenario, customer
-shouldn't be using the destination account yet. Then, after I switch MX,
-I'd do a final sync based on date. The big advantage of this is, you
-get an exact sync.
-
-Easier, however, is method 1, where I sync based on message ID, this
-way, I can use the same syntax before and after MX change, with no
-worries. Only disadvantage, you might not get an exact sync, there
-might be some difference in terms of total emails in folders, due to
-duplicates, emails that had same message ID in source server.
-
-* Method 1 - sync based on message ID, can use same syntax before and
-  after MX change
+This type of copy keeps destination exactly like source (except, it
+doesn't copy duplicates), so use it ONLY before changing your MX to
+the new server, as it will delete whatever is in destination which
+isn't in source.
 
   imapsync --host1 imap.myisp.com --user1 user@domain.com --password1 pwd \
            --host2 imap.myisp.com --user2 user@domain.com --password2 pwd \
-		   --addheader
+           --subscribeall --delete2 --delete2folders --addheader
 
 Note: add header adds message ID when it doesn't exist.
 
@@ -46,57 +36,142 @@ This syntax can also be used to sync different source accounts to one
 same destination account, simply execute it as many times as desired,
 switching source user (user1).
 
-* Method 2 - exact sync source do destination, then sync based on date
-  after MX change
+If you have a list of accounts to sync, use this:
+https://imapsync.lamiral.info/examples/sync_loop_unix.sh
 
-My first goal is to have an exact sync of an account from
-current/source host to the new/destination host and be able to sync
-several times. The --useuid parameter is very important for that
-purpose. This is what I use:
+Adjust the parameters according to the original syntax which I
+explained above.
 
-  imapsync --host1 imap.gmail.com --user1 user@domain.com --password1 pwd --ssl1 \
-           --host2 imap.myisp.com --user2 user@domain.com --password2 pwd --ssl2 \
-		   --useuid --delete2 --delete2folders 
+The Sync loop Unix script now outputs an error log, which can be
+useful.
 
-This makes it so imap.myisp.com (destination) is an exact copy of the
-account at imap.gmail.com (source). This is not a problem, since the
-user is not using the new host yet. ]You can check Imapsync log files
-and surely you will see the final difference should be 0. Check also
-for any possible errors in the log (search for "error").
+We also have shell scripts to check for errors, such as:
 
-The second goal is to lower the TTL (ex: 5 min) for the host
-associated with the MX record, in the domain's DNS server. Let's say
-customer has a host mail which his MX points to, with a high TTL
-(usually 1 hour). Lower it to 5 min so that, when you change the MX,
-it propagates faster.
+- Accounts over quota
+
+grep -R "EXIT_OVERQUOTA" *.txt
+
+- Login error
+
+grep -R "Error login" *.txt
+
+- Check just critical errors
+
+egrep 'EXIT_AUTHENTICATION_FAILURE|EXIT_CONNECTION_FAILURE|EXIT_TLS_FAILURE|EXIT_OVERQUOTA' *.txt
+
+- Check all sorts of errors (can be too much info)
+
+grep 'Exiting with return value' *.txt | grep -v 'Exiting with return value 0'
+
+Also, I like to check log file sizes and compare account sizes on
+source and destination.
+
+--
+
+Back to it.
+
+After you do a full sync, check that everything is consistant and
+let's proceed to the final stage.
+
+Lower the TTL (ex: 5 min) for the host associated with the MX record,
+in the domain's DNS server. Let's say customer has a host mail which
+his MX points to, with a high TTL (usually 1 hour). Lower it to 5 min
+so that, when you change the MX, it propagates faster.
 
 When comes time to switch over to the new host, do a final sync with
 above syntax, before changing the MX. Then, change the MX and tell
 your users to start using exclusively the new host.
 
 A few hours after the MX change, we will run Imapsync again. We have
-to start preserving emails users move or flag in the new host, which
+to start preserving emails users move or flags in the new host, which
 they started using, so we can't do an exact sync anymore.
 
-The best solution for me was to Sync any new emails (maxage:1) from
-source (that could arrive in source even after MX change, due to
-cache) and delete such emails from source server. This way, customer's
-mailbox is still intact on the source server, except new emails, which
-get synced to new server and deleted from source.
+One simple solution is the same syntax as before, but without
+--delete2 and --delete2folders.
 
-  imapsync --host1 imap.gmail.com --user1 user@domain.com --password1 pwd --ssl1 \
-           --host2 imap.myisp.com --user2 user@domain.com --password2 pwd --ssl2 \
-           --folder INBOX --useuid --noexpungeaftereach --skipemptyfolders --maxage 1 --delete1
+So:
 
-I personally prefer to keep a copy of users box intact in source, but
-if that's not an issue for you, you can remove --folder INBOX and even
---maxage, but then, all emails in source will be deleted. You can use
---maxage 1 with --delete1, however, for all folder (without specifying
---folder INBOX), so only any new email that arrives at source is
-copied to destination and deleted from source.
+  imapsync --host1 imap.myisp.com --user1 user@domain.com --password1 pwd \
+           --host2 imap.myisp.com --user2 user@domain.com --password2 pwd \
+           --subscribeall --addheader
 
-My next goal was to automate the process, so I followed this advice:
-http://imapsync.lamiral.info/examples/sync_loop_unix.sh
+The only issue with that syntax is, it would simply copy what is in
+source that is not in destination. For example, user might have moved
+emails to some sub-folder in host2. However, if you do this once, one
+hour after MX change, for example, it's comprehensible that
+destination is not exactly as the source anymore.
+
+What we've been doing, especially in the case of huge migrations,
+since our destination server auto-archives emails in sub-folders and
+changes the hierarchy, is syncing just the last day or few days of
+email and just of INBOX.
+
+So we do the sync initially explained in this doc, before the MX
+change. Then we auto-archive accounts and after auto-archiving, we
+change MX and do a final sync using maxage.
+
+Something like this:
+
+  imapsync --host1 imap.myisp.com --user1 user@domain.com --password1 pwd \
+           --host2 imap.myisp.com --user2 user@domain.com --password2 pwd \
+           --subscribeall --addheader --maxage 1
+
+If your last full sync was 2 days ago, then use --maxage 2
+
+Goal is to get all the new emails (2 last days) from source.
+
+Another alternative for syncing after the MX change, which Lamiral
+suggests, involves deleting emails from source from the last day.  I
+prefer the method explained above. Anyway, here is this other
+scenario...
+
+Something like this:
+
+ imapsync --host1 imap.gmail.com --user1 user@domain.com --password1 pwd --ssl1 \
+          --host2 imap.myisp.com --user2 user@domain.com --password2 pwd --ssl2 \
+          --folder INBOX --noexpungeaftereach --skipemptyfolders --maxage 1 --delete1
+		   
+This way, any new email that arrived at source is copied to
+destination and deleted from source.  Without the --delete1 option,
+messages moved on host2 by the user are coming back from host1, and
+flags changed on host2 are also reset to what they are on host1.
+
+However, as I wrote before, if you run it just once without --delete1,
+perhaps not that bad.
+
+*** Other tips for ISPs
+
+1) Sync entire account into 1 folder of another account
+
+  imapsync --host1 xxx --user1 user1@domain.com --password1 secret1 --ssl1 \
+           --host2 yyy --user2 user2@domain.com --password2 secret2 --ssl2 \
+           --addheader --subfolder2 "otheraccountfolder" --delete2 --delete2foldersonly /otheraccountfolder/
+
+Above, user must not be using such destination folder in destination
+account until you finish syncing.
+
+I've used it without --delete2 and --delete2foldersonly, as it caused
+problems for us, of that folder being unsubscribed in destination...
+Don't think it should be a problem as this copy is done usually
+without using really using the folder in host2 until you instruct him
+to.
+
+Another nice scenario, sync just one folder hierarchy of an account to
+another folder of another account.
+
+--subfolder1 test --subfolder2 test2 --delete2 --delete2foldersonly /test2/ --addheader
+
+Also, very useful is to be move emails older than one year, from an
+account to another one, to offload the source account.
+
+imapsync ... --subfolder2 "account_user1" --addheader --minage 365 --delete1
+
+So everything older than 365 days is deleted from source. You can even
+specify same account for source and recipient, if you want to "delete
+older than x days" for the account itself, without moving old emails
+to another account.
+
+2) Translate folder names
 
 I also ended up requiring a regex to translate folder names. On the
 old server (Gmail), Sent items were in a folder called [Gmail]/E-mails
@@ -110,17 +185,54 @@ So this was added:
   --regextrans2 "s,Lixeira,Trash," \
   --regextrans2 "s,Rascunhos,Drafts,"
 
-*** Other cenarios
+Option --automap is also good to do most of the translate job.
 
-- Sync entire account into 1 folder of another account
+3) Folders to exclude, to speed up syncs, etc.
 
-  imapsync --host1 xxx --user1 user1@domain.com --password1 secret1 --ssl1 \
-           --host2 yyy --user2 user2@domain.com --password2 secret2 --ssl2 \
-		   --useuid --subfolder2 otheraccountfolder --delete2 --delete2foldersonly /otheraccountfolder/
+  imapsync ... --exclude "^Backups" --exclude "(?i)spam" --exclude "(?i)junk" --exclude "(?i)trash" --exclude "(?i)lixeira" --exclude "(?i)Deleted Items"
+
+4) Delete older than 1 year from an account.
+
+Use the account itself as source and destination and use:
+
+  imapsync ... --minage 365 --delete1
+
+5) Check size of folders in an account. Use Imapsync with same account
+   as source and destination, with this parameter:
+
+  imapsync ... --justfoldersizes
+
+
+6) If you have problems copying messages that have labels which are
+   not supported, there are a few ways out.
+
+We use Dovecot and it doesn't support a series of labels, except the
+standard ones. By default, imapsync won't copy messages that have
+labels not supported by the recipient server. For that reason, I now
+always use --filterbuggyflags and tell customers that not all labels
+are copied.
+
+There are other alternatives, such as replacing problematic flags with
+other names or copying just standard flags, detailing which ones to
+copy, however, --filterbuggyflags was the easiest solution, which
+worked fine for us.
+
+7) Definitely worth reading about dupes, here
+https://imapsync.lamiral.info/FAQ.d/FAQ.Duplicates.txt, if you have
+problems.  Always try to check, even if by sampling, if dupes
+occur. In a big migration, when you do several passes, try to check a
+few accounts.
+
+The one issue we had with many duplicated emals in the destinatio, was
+solved with the first solution in the above FAQ, using this syntax,
+when copying from source to the destination, hence effectively
+removing the dupes.
+
+  imapsync ... --useheader "Message-Id" --delete2duplicates
+
+Note that --delete2duplicates is on when --delete2 is used, unless
+--syncduplicates is used or --nodelete2duplicates is used.
 
-Above is based on message UID, advantage of no dupes, however, user
-must not be using such destination folder in destination account until
-you finish syncing.
 
 =======================================================================
 =======================================================================

FAQ.d/FAQ.IceWarp.txt

@@ -0,0 +1,23 @@
+#!/bin/cat
+$Id: FAQ.IceWarp.txt,v 1.1 2020/02/25 10:49:07 gilles Exp gilles $
+
+This document is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.IceWarp.txt
+
+=======================================================================
+              Imapsync tips for IceWarp accounts.
+=======================================================================
+
+
+=======================================================================
+Q. How to migrate from or to IceWarp?
+
+
+R. Use:
+
+  imapsync ...  --noabletosearch
+
+The option --noabletosearch is needed only if options --maxage or --minage
+are used. I don't know which --search commands could work, maybe none.
+

FAQ.d/FAQ.James.txt

@@ -0,0 +1,34 @@
+#!/bin/cat
+$Id: FAQ.James.txt,v 1.2 2022/04/06 10:28:16 gilles Exp gilles $
+
+This documentation is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.James.txt
+
+
+=======================================================================
+      Imapsync tips for James. Specific issues and solutions.
+=======================================================================
+
+James: JAVA  Apache Mail Enterprise Server
+
+Questions answered in this FAQ are:
+
+Q. How to migrate to Dovecot with an admin/MasterUser account?
+
+
+=======================================================================
+Q. How to migrate to James with an admin/MasterUser account?
+
+R. James supports impersonation and is set up via the administratoId key 
+field stored in <install>/conf/usersrepository.conf
+
+There are multiple user repository definitions, each using 
+a different key:value pair to define administratorId
+
+https://james.apache.org/server/config-users.html
+
+(Thanks to From Rob Davis for this Q/R)
+
+=======================================================================
+=======================================================================

FAQ.d/FAQ.Kerio.txt

@@ -0,0 +1,46 @@
+#!/bin/cat
+$Id: FAQ.Kerio.txt,v 1.3 2022/04/05 13:48:52 gilles Exp gilles $
+
+This document is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Kerio.txt
+
+=======================================================================
+              Imapsync tips for Kerio accounts.
+=======================================================================
+
+
+
+
+=======================================================================
+Q. How to migrate from Kerio Connect to XXX
+
+R. No special options required. 
+See also:
+http://linux-france.tk/prj/imapsync_list/msg01756.html
+http://web.archive.org/web/20140224041102/https://www.safetynet-it.com/it-support/mac-kerio-server-to-microsoft-exchange-2010-migration-1/
+http://web.archive.org/web/20121125074756/http://www.safetynet-it.com/it-support/mac-kerio-server-to-microsoft-exchange-2010-migration-2/
+
+=======================================================================
+Q. How to use Kerio master authentication with imapsync?
+
+R. Example:
+
+Kerio at host1
+
+  imapsync ... --user1 USERNAME --password1 MASTERPASSWORD --authmech1 X-MASTERAUTH 
+
+Kerio at host2
+
+  imapsync ... --user2 USERNAME --password2 MASTERPASSWORD --authmech2 X-MASTERAUTH 
+  
+See also:
+https://github.com/imapsync/imapsync/pull/53
+
+If you get the error
+"Host failure: Error login on ... auth [X-MASTERAUTH]: 2 BAD AUTHENTICATE Unknown authentication method"
+
+Then admin authentication may not be activated yet. See:
+https://forums.gfi.com/index.php?t=msg&th=28458&goto=118447&#msg_118447
+
+=======================================================================

FAQ.d/FAQ.Logfile.txt

@@ -0,0 +1,41 @@
+$Id: FAQ.Logfile.txt,v 1.2 2021/07/23 12:27:31 gilles Exp gilles $
+
+This documentation is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Logfile.txt
+
+
+======================================================================
+                       Imapsync log file
+======================================================================
+
+Questions answered in this FAQ are:
+
+Q. I have these errors "could not be fetched", like this one:
+Err 1/1: - msg INBOX/17219 {0} S[36592] F[] I[16-Jul-2021 12:04:15 +0000] could not be fetched
+What can I do about it?
+
+Now the questions again with their answers.
+
+======================================================================
+Q. I have these errors "could not be fetched", like this one:
+Err 1/1: - msg INBOX/17219 {0} S[36592] F[] I[16-Jul-2021 12:04:15 +0000] could not be fetched
+What can I do about it?
+
+
+R. Explanation of a log line:
+
+- msg INBOX/17219 {0} S[36592] F[] I[16-Jul-2021 12:04:15 +0000]
+
+INBOX    = folder
+17219    = UID
+{0}      = got 0 bytes 
+S[36592] = should have got 36592 bytes (RFC size announced)
+F[]      = Flags, none here
+I[16-Jul-2021 12:04:15 +0000] = Internal date
+
+Usually this error doesn't appear on next runs, at least not for the 
+same messages. So rerun imapsync until all messages are synced.
+
+======================================================================
+======================================================================

FAQ.d/FAQ.MailEnable.txt

@@ -0,0 +1,37 @@
+#!/bin/cat
+$Id: FAQ.MailEnable.txt,v 1.2 2020/07/07 10:45:56 gilles Exp gilles $
+
+This document is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.MailEnable.txt
+
+
+=======================================================================
+            Imapsync tips for MailEnable imap server
+=======================================================================
+
+
+
+======================================================================
+Q: Syncing to MailEnable 10.31 and after
+R: No special imapsync option is required, default is ok.
+
+======================================================================
+Q: Syncing from MailEnable 1.75
+R: Use options:
+
+    imapsync ... --sep1 "/" --prefix1 ""
+
+======================================================================
+Q: Syncing from MailEnable 2.2
+R: Use options:
+
+    imapsync ... --sep1 "." --prefix1 ""
+
+======================================================================
+Q: Syncing to MailEnable until MailEnable release 10.31 not included
+R: Use options:
+
+    imapsync ... --sep2 / --prefix2 "" --addheader --messageidnodomain --syncflagsaftercopy
+
+======================================================================

FAQ.d/FAQ.Massive.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Massive.txt,v 1.24 2018/12/30 21:28:03 gilles Exp gilles $
+$Id: FAQ.Massive.txt,v 1.34 2022/07/14 16:00:23 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -14,7 +14,7 @@ Questions answered here are:
 
 Q. How long will take the whole migration?
 
-Q. I need to migrate hundred accounts, how can I do?
+Q. I need to migrate hundreds of accounts, how can I do that?
 
 Q. I have to migrate 500k users using 400 TB of disk space. 
    How do I proceed? How about speed?
@@ -23,30 +23,79 @@ Q. How to determine where is the bottleneck in an imapsync process?
 
 Q. Can I run several instances of imapsync in parallel on a Windows host?
 
-Q. I run multiple imapsync applications at the same time then get a
+Q. I run multiple imapsync applications at the same time and then get a
    warning "imapsync.pid already exists, overwriting it".
    Is this a potential problem when trying to sync multiple 
    IMAP account in parallel?
 
 =======================================================================
-Q. How long will take the whole migration?
+Q. How long will take the whole migration? 
 
-R. Consider the mean imapsync transfer rate to be around 340 Kbytes/s,
-   ie, 2.8 Mbps, no matter the local link bandwidth. 
-   It's a mean, measured upon various different syncs, coming from 
-   the online interface /X.
-   
-   With 1 TB to transfer and one sync at a time it will end in 35 days.
-   At 10 transfers at a time, 1 TB will take 3.5 days.
-   Double the time because best scenario never happens.
-   Triple because, well, real world is like that.
+R1. First, you have to consider several periods. There is the global 
+    period, from when the migration process is decided to the end,
+    where all mailboxes are migrated. This global period can be divided
+    into three smaller periods.
+    
+    The first period is the analysis period: you play with the tools 
+    available, you estimate the volume to be transferred, the number of
+    accounts, you measure how long it takes for one account under
+    your context.
+    
+    The second period is what I call the presyncing period. 
+    The users are still using
+    the old accounts but nothing prevents you from starting to sync
+    the old accounts, as they are, to the new accounts.
+    With tons of gigabytes to transfer, this period may be the longest one. 
+    There is nothing more than launching the presyncs and monitoring them 
+    until the round is finished.
+    
+    The last period is the final sync period where only the last 
+    mailboxes' changes  need to be synced before switching
+    the users to their new mailboxes.
+
+R2. To estimate the presyncing period, consider the mean imapsync transfer
+    rate to be around 340 Kbytes/s, ie, 2.8 Mbps, no matter the local 
+    link bandwidth.
+    It's a mean, measured upon various syncs, coming from 
+    the online service /X where the network card flow rate is 200 Mbps 
+    (Rx 100 Mbps, Tx 100 Mbps) and the provider bandwidth is also 200 Mbps.
+    The maximum global rate seen is 22 MiB/s (176 Mbps).
    
+    At 340 Kbytes/s,
+    1 TB to transfer and one sync at a time will end in 35 days (1024^3/340/3600/24).
+    At  10 transfers at a time, 1 TB will take 3.5 days.
+    At 100 transfers at a time, 1 TB will take 8 hours.
+    Double the time because the best scenario never happens.
+    Triple the time because, well, the real world is like that.
+
+R3. Another way to better evaluate the end of the presyncing period can 
+    be based on your actual data. Just apply a simple rule of three 
+    on the mailboxes already migrated to estimate the global end.
+    If it took X hours to finish Y% of the mailboxes, then it
+    will take 100*X/Y hours to finish 100% of the mailboxes.
+
+    Following the same idea but using mathematical garbage, 
+    the ETA can be estimated like this: 
+    t_0      = time of global start (the start of the first presync)
+    t_now    = time of now.
+    Nb_total = total number of mailboxes to be migrated.
+    Nb_now   = number of mailboxes already migrated.
+    
+    then
+    
+    ETA = t_end = (t_now - t_0) * (Nb_total / Nb_now) + t_0
+
+
+R4. To estimate the last period, the final sync, just rerun a complete 
+    presync, ie, resync all the mailboxes, the final sync should take 
+    the same amount of time.
+    
 =======================================================================
-Q. I need to migrate hundred accounts, how can I do?
+Q. I need to migrate hundreds of accounts, how can I do that?
 
 R. If you have many mailboxes to migrate, think about a little
 script program. Write a file called file.txt (for example)
-containing hosts, users and passwords on both sides.
+containing hosts, users, and passwords on both sides.
 The separator used in this example is ";"
 
 The file.txt file contains for example:
@@ -57,8 +106,8 @@ host003_1;user003_1;password003_1;host003_2;user003_2;password003_2;
 host004_1;user004_1;password004_1;host004_2;user004_2;password004_2;
 etc.
 
-Most of the times, the first column (host001_1, host002_1 ...) will
-contains the same value, the value of --host1 parameter. Same
+Most of the time, the first column (host001_1, host002_1 ...) will
+contain the same value, the value of --host1 parameter. Same
 thing for the third column (host001_2, host002_2).
 
 On Unix the shell script can be:
@@ -99,7 +148,7 @@ becomes
 host001_1;user001_1;password001_1;host001_2;user001_2;password001_2; --automap --addheader
 host002_1;user002_1;password002_1;host002_2;user002_2;password002_2; --automap --addheader
 
-With this solution, options can be added, changed or removed per account.
+With this solution, options can be added, changed, or removed per account.
 Technically those options will go in %%M in the loop body 
 
 Here is a complete Windows example ready to use: 
@@ -119,49 +168,54 @@ of sync_loop_windows.bat
 Q. I have to migrate 500k users using 400 TB of disk space. 
    How do I proceed? How about speed?
 
-R. Solution to this issue is two words: parallelism and measurements.  
+R. A good solution to this issue is two words: parallelism and measurements.  
 
-Since all mailboxes are functionnaly independent, they can be processed 
-independently, here comes parallelism, lunching several imapsync 
+Since all mailboxes are functionally independent, they can be processed 
+independently, here comes the parallelism, ie, launching several imapsync 
 processes in parallel.
 
-Meanwhile, mailboxes usually belong to the same server and syncs
+Meanwhile, mailboxes usually belong to the same server, and the syncs
 share the same imapsync host via the same bandwidth, here come 
 some limitations and bottlenecks.
 
-How many syncs can we run in parallel? here comes measurements.
+How many syncs can we run in parallel in your context?
+Here comes some measurements.
 
 1) Measure the total transfer rate by adding each one printed in each run.
    Since adding this way is not so easy, just look at the overall 
    network rate of the imapsync host.
 
-   On Linux and FreeBSD, nload is a good candidate to measure this overall
-   network rate, every 6 seconds (-t 6000), on eth0 or em0 interface, 
-   values in Kbytes (-u K):
+   On Linux and FreeBSD, the command "nload" is a good candidate to measure this overall
+   network rate. For example, to measure the rate every 6 seconds (-t 6000), 
+   on eth0 or em0 interface, with values in Kbytes (-u K), use:
    
      nload -t 6000 eth0 -u K  # Linux
      nload -t 6000 em0  -u K  # FreeBSD
 
-   During nload, press the h to see the display commands available, 
-   every single feature is useful! Press h again and try each one.
-     
    On Linux only, another very good network tool is dstat:
    
      dstat -n -N eth0 6  # Linux only (in 2018)
 
-   A excellent tool for this purpose is iftop, the following
-   command will monitor imap and imaps connexions
-   on interfce eth0, only them, and sum them up:
+   Another excellent tool to measure the network traffic is iftop.
+   The following command will monitor imap and imaps connections
+   on interface eth0, only them, and sum them up:
    
      iftop -i eth0 -f 'port imap or port imaps' -B # Linux
      iftop -i em0  -f 'port imap or port imaps' -B # FreeBSD
 
+   During iftop, press the h to see the display commands available, 
+   every single feature is useful! Press h again and try each one.
+   My preferred display combination is by typing the keys
+   t p >  
+   t  means "one line per connection"
+   p means "show port numbers"
+   > means "sort by destination"
 
 
    On Windows 8.1 Windows 10 Windows 2012 R2 Windows 2016,
    get the overall network rate with the classical
    task manager (Ctrl-Shift-Esc), there is a Performance tab
-   in it where resides a Network grap monitor.
+   in it, where resides a Network monitor.
 
    On Windows 7, get the overall network rate with the classical
    task manager (Ctrl-Shift-Esc), there is a Network tab in it.
@@ -173,7 +227,7 @@ How many syncs can we run in parallel? here comes measurements.
    transfer rate increase.
 
 3) When the total transfer rate starts to diminish, stop new launches. 
-   Note N as the number of parallel runs you got until then.
+   Note N as the number of runs in parallel you got until then.
 
 4) Only keep N-2 parallel runs for the future.
 
@@ -183,15 +237,15 @@ Q. How to determine where is the bottleneck in an imapsync process?
 
 R1. Divide and conquer.
 
-In order to detect whether host1/link1 is the bottleneck or
+To detect whether host1/link1 is the bottleneck or
 host2/link2, we have several tests to explore:
 
-1) run a sync from host1 to host1, with a host1 test account as destination.
-This way, only host1+link1 are tested, host2 is not directly concerned.
+1) run a sync from host1 to host1, with a host1 test account as the destination.
+This way, only host1 and link1 are tested, host2 is not directly concerned.
 If performances increase a lot then host2/link2 is the bottleneck.
 
-2) run a sync from host2 to host2, with a host2 test account as destination.
-This way, only host2+link2 are tested, host1 is not concerned.
+2) run a sync from host2 to host2, with a host2 test account as the destination.
+This way, only host2 and link2 are tested, host1 is not concerned.
 If performances increase a lot then host1/link1 is the bottleneck.
 
 If performances increase on both tests 1) and 2), I have no clue to explain that.
@@ -199,54 +253,56 @@ Same thing if they both decrease!
 
 R2. Isolating and overcoming bottlenecks
 
-On any process involving several mechanisms, among all elements taking
-part on the process, there is always a bottleneck.  No one knows in
+In any process involving several mechanisms, among all elements taking
+part in the process, there is always a bottleneck.  No one knows in
 advance what is the first bottleneck. The first bottleneck has to be
 determined, by measurements, not by guesses.  Once this first
 bottleneck is known and overcome then the next bottleneck has to be
 determined and overcome too, if needed. Repeat the process of looking
 for the next bottleneck and its elimination until you estimate the
 transfer rates, money costs, time spent on this, and final dates 
-are good enough to proceed the whole huge migration.
+are good enough to proceed with the whole huge migration.
 
 Possible bottlenecks: 
 
 - Throttles. 
   IMAP servers have artificial limits. 
-  For example Gmail, Office365, Exchange have throttle limits.
+  For example, Gmail, Office365, and Exchange have throttle limits.
 
 - Bandwidth. 
-  Usually available bandwidth is NOT a bottleneck.
-  Meanwhile, it can be a bottleneck on small Internet connexions.
+  Usually, the available bandwidth is not a bottleneck.
+  Meanwhile, it can be a bottleneck on small Internet connections.
   Imapsync downloads messages from host1 and upload messages to host2,
-  consider this in case the connexion are asymmetric.
+  consider this in case the connection is asymmetric.
   
-- I/O on disks. 
-  I/O are a classical bottleneck, almost always forgotten.
+- I/O, aka "Input/Output" on the disks of the imap servers.
+  The I/O on disks are a classical bottleneck, almost always forgotten.
   Unlike CPU and RAM, Input/Output performances don't improve
   very much as time goes on so it's often a bottleneck.
   To measure and overcome an I/O disk bottleneck, you need 
-  usually a direct access to host1 and host2.
+  usually direct access to host1 and host2.
   An I/O bottleneck where imapsync runs is possible if
   --usecache or --useuid is used or with very big messages.
 
 
-- RAM memory.
-  On all sides, monitor that your systems don't swap on disk,
-  because swapping memory on disks decreases performance by 
-  a factor of 20, at least.
+- RAM.
+  On all sides, monitor that your systems don't swap its 
+  running processes on disk, because swapping running processes 
+  on disks decreases performance by a factor of 20, at least.
+  It's not because the swap memory is used that your
+  system swaps processes on disk.
 
 - CPU.
   100% CPU during a whole transfer means the system is busy.
-  Usually CPU is not a problem with imapsync but it can be a problem
-  with one of the imap servers.
-  Most often CPU is not the real bottleneck, I/O are.
+  CPU can be a problem with imapsync but it can also be a problem
+  with one or both of the imap servers.
 
 Other possible bottlenecks:
 - Number of hosts available to run imapsync processes.
 - Imapsync itself.
-- Errors management.
+- Management of errors.
 - MX domains, DNS.
+- Incompetence.
 - Money.
 - Time.
 - Bad luck.
@@ -258,12 +314,12 @@ Q. Can I run several instances of imapsync in parallel on a Windows host?
 
 R. Yes!
 
-Q. Any performance issue?
+Q. Any performance issues?
 
 You have to try and check the transfer rates, sum them up to
-have a uniq numeric criteria. 
+have a unique numeric criterion. 
 There is always a limit, depending on remote imap servers 
-and the one running imapsync;
+and the one running imapsync.
 CPU, memory, Inputs/Outputs are the classical bottlenecks, 
 the worst bottleneck is the winner that sets the limit.
 
@@ -282,7 +338,7 @@ REM After that, just double-click on each batch file to launch each process
 
 
 =======================================================================
-Q. I run multiple imapsync applications at the same time then get a
+Q. I run multiple imapsync applications at the same time and then get a
    warning "imapsync.pid already exists, overwriting it".
    Is this a potential problem when trying to sync multiple 
    IMAP account in parallel?
@@ -292,7 +348,7 @@ by yourself.
 
 This file can help you to manage multiple runs by sending signals 
 to the processes (sigterm or sigkill) using their PID.
-Each run can have its own pid file with --pidfile option.
+Each run can have its pid file with --pidfile option.
 The file imapsync.pid contains the PID of the current imapsync process.
 This file is removed at the end of a normal run.
 You can safely ignore the warning if you don't use imapsync.pid file

FAQ.d/FAQ.Messages_Selection.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Messages_Selection.txt,v 1.14 2018/05/24 11:34:30 gilles Exp gilles $
+$Id: FAQ.Messages_Selection.txt,v 1.18 2021/07/04 23:05:24 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -33,7 +33,54 @@ Q. What are the selection criteria available with --search option?
 =======================================================================
 Q. What messages imapsync syncs by default?
 
-R. By default, Imapsync syncs all messages, except duplicates.
+R1. By default, Imapsync syncs all messages, except duplicates.
+    If you want to sync duplicates, add the option --syncduplicates
+
+R2. You can alter the default "all messages selected" by using one or 
+    more of the following options:
+
+Extract from the README https://imapsync.lamiral.info/README
+
+  OPTIONS/message selection
+
+     --maxsize      int  : Skip messages larger  (or equal) than  int  bytes
+     --minsize      int  : Skip messages smaller (or equal) than  int  bytes
+
+     --maxage       int  : Skip messages older than  int days.
+                           final stats (skipped) don't count older messages
+                           see also --minage
+     --minage       int  : Skip messages newer than  int  days.
+                           final stats (skipped) don't count newer messages
+                           You can do (+ zone are the messages selected):
+                           past|----maxage+++++++++++++++>now
+                           past|+++++++++++++++minage---->now
+                           past|----maxage+++++minage---->now (intersection)
+                           past|++++minage-----maxage++++>now (union)
+
+     --search       str  : Selects only messages returned by this IMAP SEARCH
+                           command. Applied on both sides.
+                           For a complete set of what can be search see
+                           https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt
+
+     --search1      str  : Same as --search but for selecting host1 messages only.
+     --search2      str  : Same as --search but for selecting host2 messages only.
+                           So --search CRIT equals --search1 CRIT --search2 CRIT
+
+     --maxlinelength int : skip messages with a line length longer than  int  bytes.
+                           RFC 2822 says it must be no more than 1000 bytes but
+                           real life servers and email clients do more.
+
+
+     --useheader    str  : Use this header to compare messages on both sides.
+                           Ex: Message-ID or Subject or Date.
+     --useheader    str    and this one, etc.
+
+     --usecache          : Use cache to speed up next syncs. Not set by default.
+     --nousecache        : Do not use cache. Caveat: --useuid --nousecache creates
+                           duplicates on multiple runs.
+     --useuid            : Use UIDs instead of headers as a criterion to recognize
+                           messages. Option --usecache is then implied unless
+                           --nousecache is used.
 
 =======================================================================
 Q. Is there a way we can specify a date range to sync emails? 
@@ -70,21 +117,27 @@ or
 Q. Is there a way we can specify an age to sync emails? 
    If yes, can you please share some examples?
 
-R. Yes, with the --maxage or the --minage option.
 
-E.1 Sync only messages less than 2 days old:
+R. Yes, with the --maxage or the --minage option. 
+
+   Have in mind that the options --maxage and --minage can NOT be 
+   combined successfully with the --search command. If the --search 
+   option is present on the command line then --maxage and --minage 
+   are ignored.
+
+Example.1 Sync only messages less than 2 days old:
 
   imapsync ... --maxage 2
 
-E.2 Sync only messages more than 2 days old:
+Example.2 Sync only messages more than 2 days old:
 
   imapsync ... --minage 2
 
-E.3 Sync only messages more than 30 days old and less than 365 days old:
+Example.3 Sync only messages more than 30 days old and less than 365 days old:
 
   imapsync ... --minage 30 --maxage 365
 
-E.4 Sync only messages less than 30 days old or more than 365 days old:
+Example.4 Sync only messages less than 30 days old or more than 365 days old:
 
   imapsync ... --maxage 30 --minage 365
 
@@ -101,9 +154,9 @@ Full explanation:
                          past|----maxage+++++minage---->now (intersection)
                          past|++++minage-----maxage++++>now (union)
 
-C.1 By default,
+Caveat.1 By default,
 
-  option --maxage is implemented as a --search SENTSINCE 
+  Option --maxage is implemented as a --search SENTSINCE 
 RFC 3501 says: SENTSINCE <date>
 Messages whose [RFC-2822] Date: header (disregarding time and
 timezone) is within or later than the specified date.
@@ -151,12 +204,17 @@ Q. Can I migrate only mails with attachments?
 
 R. Use:
 
+  imapsync ... --search "HEADER Content-Disposition attachment" 
+
+or
+
   imapsync ... --search "HEADER Content-Type multipart/mixed" 
 
 or more generally:
 
   imapsync ... --search "OR HEADER Content-Disposition attachment HEADER Content-Type multipart/mixed" 
 
+The last one doesn't work on all imap servers, not on Gmail at least (tested 2020/06/16)
 
 =======================================================================
 Q. How can I move messages marked \Deleted from all folders to
@@ -172,10 +230,34 @@ Q. What are the selection criteria available with --search option?
 
 R. The list of search criteria are listed below, an excerpt from RFC3501.
 
-http://www.faqs.org/rfcs/rfc3501.html
 
-6.4.4.  SEARCH Command
+https://tools.ietf.org/html/rfc3501#section-6.4.4
 ...
+      The SEARCH command searches the mailbox for messages that match
+      the given searching criteria.  Searching criteria consist of one
+      or more search keys.  The untagged SEARCH response from the server
+      contains a listing of message sequence numbers corresponding to
+      those messages that match the searching criteria.
+      When multiple keys are specified, the result is the intersection
+      (AND function) of all the messages that match those keys.  For
+      example, the criteria DELETED FROM "SMITH" SINCE 1-Feb-1994 refers
+      to all deleted messages from Smith that were placed in the mailbox
+      since February 1, 1994.  A search key can also be a parenthesized
+      list of one or more search keys (e.g., for use with the OR and NOT
+      keys).
+
+      In all search keys that use strings, a message matches the key if
+      the string is a substring of the field.  The matching is
+      case-insensitive.
+
+      The defined search keys are as follows.  Refer to the Formal
+      Syntax section for the precise syntactic definitions of the
+      arguments.
+
+      <sequence set>
+         Messages with message sequence numbers corresponding to the
+         specified message sequence number set.
+
       ALL
          All messages in the mailbox; the default initial key for
          ANDing.
@@ -306,6 +388,89 @@ http://www.faqs.org/rfcs/rfc3501.html
       UNSEEN
          Messages that do not have the \Seen flag set. 
 
+...
+
+search          = "SEARCH" [SP "CHARSET" SP astring] 1*(SP search-key)
+                    ; CHARSET argument to MUST be registered with IANA
+...
+
+search-key      = "ALL" / "ANSWERED" / "BCC" SP astring /
+                  "BEFORE" SP date / "BODY" SP astring /
+                  "CC" SP astring / "DELETED" / "FLAGGED" /
+                  "FROM" SP astring / "KEYWORD" SP flag-keyword /
+                  "NEW" / "OLD" / "ON" SP date / "RECENT" / "SEEN" /
+                  "SINCE" SP date / "SUBJECT" SP astring /
+                  "TEXT" SP astring / "TO" SP astring /
+                  "UNANSWERED" / "UNDELETED" / "UNFLAGGED" /
+                  "UNKEYWORD" SP flag-keyword / "UNSEEN" /
+                    ; Above this line were in [IMAP2]
+                  "DRAFT" / "HEADER" SP header-fld-name SP astring /
+                  "LARGER" SP number / "NOT" SP search-key /
+                  "OR" SP search-key SP search-key /
+                  "SENTBEFORE" SP date / "SENTON" SP date /
+                  "SENTSINCE" SP date / "SMALLER" SP number /
+                  "UID" SP sequence-set / "UNDRAFT" / sequence-set /
+                  "(" search-key *(SP search-key) ")"
+
+seq-number      = nz-number / "*"
+                    ; message sequence number (COPY, FETCH, STORE
+                    ; commands) or unique identifier (UID COPY,
+                    ; UID FETCH, UID STORE commands).
+                    ; * represents the largest number in use.  In
+                    ; the case of message sequence numbers, it is
+                    ; the number of messages in a non-empty mailbox.
+                    ; In the case of unique identifiers, it is the
+                    ; unique identifier of the last message in the
+                    ; mailbox or, if the mailbox is empty, the
+                    ; mailbox's current UIDNEXT value.
+                    ; The server should respond with a tagged BAD
+                    ; response to a command that uses a message
+                    ; sequence number greater than the number of
+                    ; messages in the selected mailbox.  This
+                    ; includes "*" if the selected mailbox is empty.
+
+seq-range       = seq-number ":" seq-number
+                    ; two seq-number values and all values between
+                    ; these two regardless of order.
+                    ; Example: 2:4 and 4:2 are equivalent and indicate
+                    ; values 2, 3, and 4.
+                    ; Example: a unique identifier sequence range of
+                    ; 3291:* includes the UID of the last message in
+                    ; the mailbox, even if that value is less than 3291.
+
+sequence-set    = (seq-number / seq-range) *("," sequence-set)
+                    ; set of seq-number values, regardless of order.
+                    ; Servers MAY coalesce overlaps and/or execute the
+                    ; sequence in any order.
+                    ; Example: a message sequence number set of
+                    ; 2,4:7,9,12:* for a mailbox with 15 messages is
+                    ; equivalent to 2,4,5,6,7,9,12,13,14,15
+                    ; Example: a message sequence number set of *:4,5:7
+                    ; for a mailbox with 10 messages is equivalent to
+                    ; 10,9,8,7,6,5,4,5,6,7 and MAY be reordered and
+                    ; overlap coalesced to be 4,5,6,7,8,9,10.
+
+
+See also the excellent "Introduction to IMAP" by Nick Babcock
+https://nbsoftsolutions.com/blog/introduction-to-imap
+
+
+
+See also "WITHIN Search Extension to the IMAP Protocol"
+https://tools.ietf.org/html/rfc5032
+
+...
+   search-key =/ ( "OLDER" / "YOUNGER" ) SP nz-number
+                  ; search-key defined in RFC 3501
+
+4.  Example
+
+   C: a1 SEARCH UNSEEN YOUNGER 259200
+   S: a1 * SEARCH 4 8 15 16 23 42
+
+   Search for all unseen messages within the past 3 days, or 259200
+   seconds, according to the server's current time.
+   
 =======================================================================
 =======================================================================
 

FAQ.d/FAQ.Messages_Too_Big.txt

@@ -0,0 +1,139 @@
+#!/bin/cat
+$Id: FAQ.Messages_Too_Big.txt,v 1.8 2021/12/16 11:44:26 gilles Exp gilles $
+
+This document is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Too_Big.txt
+
+=======================================================================
+              Dealing with too big messages with Imapsync.
+=======================================================================
+
+Questions answered in this FAQ are:
+
+Q. Imapsync reports errors like these ones:
+   "could not append...Maximum size of appendable message has been exceeded"
+   "could not append...maximum message size exceeded"
+   "The most frequent error is ERR_APPEND_SIZE"
+   What does it mean?
+
+Q. Does imapsync limits big messages?
+
+Q. What are the well known servers messages size limits?
+
+Q. What can I do to transfer messages bigger than what allows the 
+   imap destination server?
+
+Q. The solution --truncmess xxxx doesn't work with Gmail. 
+   What can I do?
+
+
+Now the questions again with their answers.
+
+=======================================================================
+Q. Imapsync reports errors like these ones:
+   "could not append...Maximum size of appendable message has been exceeded"
+   "could not append...maximum message size exceeded"
+   "The most frequent error is ERR_APPEND_SIZE"
+   What does it mean?
+
+R. It means the message couldn't be transferred to the destination
+   account because it is too big, imapsync tries to APPEND it to the
+   server (that's IMAP jargon) but the server refuses it by saying
+   either "NO. Maximum size of appendable message has been exceeded"
+   or "NO. maximum message size exceeded".
+
+
+=======================================================================
+Q. Does imapsync limits big messages?
+
+R. No. Imapsync synchronizes messages no matter their sizes.
+
+When a message is copied to the destination server, the destination
+server accepts it or not. It does it as a whole, it takes the full
+message or nothing. The destination server can refuse a too big
+message, it then triggers an APPEND error message in imapsync.
+
+So far, December 2021, the biggest message transferred by imapsync on
+the online /X service is 1.51 GiB. Who can top that?
+
+
+=======================================================================
+Q. What are the well known servers messages size limits?
+
+   * Gmail message size limit is 35 MB. 
+     Gmail announces APPENDLIMIT=35651584 (June 2021).
+   
+   * Office365 message size limit is 45 MB by default
+     and it can be upped by configuration. See 
+     https://imapsync.lamiral.info/FAQ.d/FAQ.Office365.txt
+     Q. I see "NO Maximum size of appendable message has been exceeded"
+
+   * Exchange message size limit is 10 MB by default
+     and it can be upped by configuration. See
+     https://imapsync.lamiral.info/FAQ.d/FAQ.Exchange.txt
+     Q. I see "NO Maximum size of appendable message has been exceeded"
+
+   * Dovecot message size limit is variable. It is defined
+     by the sysadmin. For example, quota_max_mail_size = 100M
+     https://doc.dovecot.org/configuration_manual/quota/
+     
+   * Usually, any IMAP server has a message size limit, defined 
+     by the sysadmin. You discover it sooner or later
+
+=======================================================================
+Q. What can I do to transfer messages bigger than what allows the 
+   imap destination server?
+
+R1. A solution is ask imapsync to truncate the message to the maximum
+size allowed before copying it to host2 by using the option
+--truncmess.
+
+That's a little dirty but it's ok if you prefer having a truncated
+message than no message at all. The attachments after the truncation
+won't be available but everything before should be ok. It may also
+depend on the software tool reading the email message.
+
+
+To truncate messages to 10 MB, use: 
+
+  imapsync ... --truncmess 10_000_000
+
+
+R2. A not-so-dirty solution would be to transform the big messages,
+replace the attachments with links to the attachments stored
+elsewhere.  Well, it's not done and it's too much complicated compared
+to a simpler solution which is to allow bigger messages on the
+destination account.
+
+As time goes on, disk space is cheaper and cheaper.  Don't make the
+new user account, the destination account, worse than their previous
+account: buy disks.
+
+Monitor and rule your hardware resources, serve your users,
+not the contrary.
+
+
+=======================================================================
+Q. The solution --truncmess xxxx doesn't work with Gmail. 
+   What can I do?
+
+R. Gmail currently limits messages size up to 35651584 bytes, aka 34
+  MiB (June 2021) but imapsync knows it so it avoid to transfer these
+  big messages. Solution, use:
+
+  imapsync ... --truncmess 35_651_584 --appendlimit 1_000_000_000_000
+
+Explanation. By default, imapsync checks if the host2 imap server
+announces a message size limit with "APPENDLIMIT=xxxx" in its
+CAPABILITY response, like Gmail does.  Imapsync takes this APPENDLIMIT
+value to set automatically the --maxsize option, unless --maxsize is
+already set on the command line and has a smaller value than the
+APPENDLIMIT value.
+
+1_000_000_000_000 is one TB, I guess you don't have any message of
+that size yet.
+
+=======================================================================
+=======================================================================
+

FAQ.d/FAQ.Migration_Plan.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Migration_Plan.txt,v 1.5 2019/01/11 13:13:55 gilles Exp gilles $
+$Id: FAQ.Migration_Plan.txt,v 1.9 2021/12/19 10:00:32 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -8,23 +8,23 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Migration_Plan.txt
 
 =====================================================================
    Imapsync. Suggestions for a good, low impact on users, 
-   well executed email migration plan.
+   well-executed email migration plan.
 =====================================================================
 
-There is two main different scenarios depending on the response to the
-following question:
+There are two main different scenarios. Choosing which one fits your
+context depends on the response to the following question:
 
-Will the imap software tools used by the users use the exact same
+Will the imap software tools used by the users use the same
 credentials triplet for both imap servers, the old server host1 and
 the new server host2?
 
 The credentials triplet is hostname/username/password.
 
-If the answer is yes, ie, clients email tools use the exact same
-triplet credentials, then it is possible to perform a migration
-without changing anything on the users side. This may be a very time
-saving option. But it's a rare condition so I'll describe this
-scenario later in this document.
+If the answer is yes, ie, clients' email tools use the same triplet
+credentials, then it is possible to perform a migration without
+changing anything on the users' side. This may be a very time-saving
+option. But it's a rare condition so I'll describe this scenario later
+in this document.
 
 
 =====================================================================
@@ -40,101 +40,124 @@ Classical scenario, credentials triplets are different on both sides
    users are already playing with the new mailboxes on host2, don't
    follow this scenario.
 
- * Presync all the mailboxes from the old server host1 to the new
-   server host2.  Presyncs can usefully be done with --delete2 in
-   order to get an exact sync. But never use the option --delete2 once
+ * Pre-synchronize all the mailboxes from the old server host1 to the
+   new server host2. If an imap server name is going to change its IP
+   address, then don't use this name, use a name that will always
+   match the same imap servers, or use their IP addresses.
+   
+   Pre-synchronizations can usefully be done with --delete2 to get an
+   exact synchronization. But never use the option --delete2 once the
    users have started to play with their new account on host2, their
-   play will be lost on the next presync, or when the MX is changed,
-   since INBOX will start to receive new messages that are not on
-   host1.
+   play will be lost on the next synchronization. Don't use --delete2
+   either when the MX is changed since INBOX will start to receive new
+   messages that are not on host1 and then removing them is not a good
+   idea.
 
  * Decide a migration day/hour.
 
- * Repeat the presyncs (with the --delete2 options) daily until the
-   migration hour. This repeated process will show how long should
-   take the last sync.
+ * Repeat the pre-synchronizations (with the --delete2 options) daily
+   until the migration hour. This repeated process will show how long
+   should take the last synchronization.
 
  * At the migration hour, cut access to the users to the old server
    host1, if you can. Or tell them to not use it anymore.
 
- * Do a last presync exactly like previous ones.
+ * Do the last pre-synchronization exactly like the previous ones.
 
  * Change the MX, the new messages should start to arrive in the new
    imap server host2.
 
- * Wait the TTL value, aka 5 minutes. Now, new messages should 
-   not arrive to the old server host1.
+ * Wait for the TTL value, aka 5 minutes. Now, new messages should 
+   not arrive at the old server host1.
 
  * Tell the users that the old imap server host1 is down and no 
    longer available.
 
- * Do a postsync. A postsync is a sync with the following options 
-   --maxage 1 --delete1 --folder INBOX
+ * Do a post-synchronization. A post-synchronization is a run with the
+   following options: --folder INBOX --delete1 --maxage 1
 
-   This postsync will move the last new messages arrived on host1 to
-   host2 during the TTL interval and delete them on host1. Do not use
-   the option --delete2 in a postsync.
+   This post-synchronization will copy the messages arrived in the
+   last day (--maxage 1) in the folder INBOX (--folder INBOX) on the
+   source account, to the destination account. It will also delete
+   them on host1 (--delete1). It's --delete1, it's not --delete2.
+   Remember, do not use the option --delete2 in a post
+   synchronization, as users won't appreciate seeing their newly
+   arrived messages disappear because of you.
 
  * Give access to new accounts to the users with their new credential
    triplet hostname/username/password.  If the way to contact users is
-   email then you should give this long before shutting down the old
-   server.
+   by email then you should give them the new credentials long before
+   shutting down the old server.
 
  * Migration done.
 
  * In case there are still messages arriving at the old imap server
-   host1, you can perform more postsyncs, ie, syncs every day 
-   with the options:
-   --maxage 1 --delete1 --folder INBOX.
+   host1, you can perform more post-synchronizations, ie, runs every
+   day with the options: --maxage 1 --delete1 --folder INBOX
 
+ * Increase the TTL of the MX back to its previous value, usually
+   24 hours, 86400 seconds. You don't want all your email system
+   to break down completely when your DNS are not available 
+   temporarily, keeping dns values in cache for a 24h is a savvy
+   practice.
 
 =====================================================================
 Lucky scenario, credentials triplets are the same on both sides
 =====================================================================
  
  * Decrease the TTL of the MX, as well as the imap hostname resolution,
-   to 5 minutes (or even less). See FAQ.TTL.txt to understand why.
+   to 5 minutes (or even less). The document FAQ.TTL.txt explains why.
 
  * Create the new mailboxes on the destination server host2.
  
- * Presync all the mailboxes from the old host1 to the new server host2, 
-   using different names that the ones used by the imap software 
-   clients (use their IP for example).
-   Presyncs have to be done with --delete2 but never use --delete2 
-   once users have started playing with their new account on host2.
+ * Pre-synchronize all the mailboxes from the old host1 to the new
+   server host2, using different names than the ones used by the imap
+   software clients (use their IP for example).  Presyncs have to be
+   done with --delete2 but never use --delete2 once users have started
+   playing with their new account on host2.
  
  * Decide a migration day/hour.
 
- * repeat the presyncs (with the --delete2 options) daily until the 
-   migration hour. This repeated process will show how long should 
-   take the last sync.
+ * Repeat the pre-synchronizations (the runs with the --delete2
+   options) daily until the migration hour. This repeated process will
+   show how long should take the last sync.
 
  * At the migration hour, cut access to the users to the old server.
    You can do this by changing the imap host1 hostname to a non-imap 
    server for example, or by changing their password on host1.
 
- * Do a last sync exactly like the presyncs, not using the imap hostname.
+ * Do the last run exactly like the pre-synchronizations.
 
  * Change also the MX resolution, the new messages should start 
    to arrive in the new imap server very soon.
  
- * Wait the TTL value, aka 5 minutes. Now, new messages should 
-   not arrive to the old server host1.
+ * Wait for the TTL value, aka 5 minutes. Now, new messages should 
+   not arrive at the old server host1.
  
- * Do a postsync. A postsync is a sync with the following options 
-   --maxage 1 --delete1 --folder INBOX
+ * Do a post-synchronization. A post-synchronization is a run with the
+   following options: --folder INBOX --delete1 --maxage 1
 
-   This postsync will move the last new messages arrived on host1 to
-   host2 during the TTL interval and delete them on host1. Do not use
-   the option --delete2 in a postsync.
+   This post-synchronization will copy the messages arrived in the
+   last day (--maxage 1) in the folder INBOX (--folder INBOX) on the
+   source account, to the destination account. It will also delete
+   them on host1 (--delete1). It's --delete1, it's not --delete2.
+   Remember, do not use the option --delete2 in a post
+   synchronization, as users won't appreciate seeing their newly
+   arrived messages disappear because of you.
 
- * Shutdown the old imap server.
+ * Shut down the old imap server.
 
  * Change the user imap hostname resolution from the old IP of host1
    to the IP of the new imap server host2.
 
  * Migration done.
 
+ * Increase the TTL of the MX back to its previous value, usually
+   24 hours, 86400 seconds. You don't want all your email system
+   to break down completely when your DNS are not available 
+   temporarily, keeping dns values in cache for a 24h is a savvy
+   practice.
+
  =======================================================================
  =======================================================================
  

FAQ.d/FAQ.Office365.txt

@@ -0,0 +1,686 @@
+
+$Id: FAQ.Office365.txt,v 1.38 2022/06/22 17:07:35 gilles Exp gilles $
+
+This documentation is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Office365.txt
+
+
+======================================================================
+            Imapsync tips for Office365.
+======================================================================
+
+Questions answered in this FAQ are:
+
+Q. Can I use imapsync to transfer from or to Office365 accounts?
+
+Q. Can imapsync work for users that are administrators for 
+an Office 365 domain? (quick answer: no)
+
+Q. Does imapsync support OAUTH2 authentication for Office365 accounts?
+
+Q. How to sync from Office365 to XXX?
+
+Q. How to sync from XXX to Office365
+
+Q. For Office365 I have double and triple-checked the username and
+password spelling but I still get a "LOGIN failed". Any clue?
+
+Q. How can I access an Office365 shared mailbox?
+
+Q. How can I use a shared account as a backup account for several
+mailboxes and so avoid spending too many dollars in backup accounts?
+
+Q. Office365 fails with "User is authenticated but not connected".
+
+Q. I see "NO Maximum size of appendable message has been exceeded"
+   What can I do with that?
+
+Q. Every single mail synced to exchange online owns the 
+   category $MDNSent after migration. How can I avoid this?
+
+Q. The imap connection to Office365 is not working very efficiently,
+   is there a solution to fix that?
+
+Q. Office365 throttles the sync and says: 
+   "Request is throttled. Suggested Backoff Time: 299961 milliseconds".
+   What can I do with that?
+
+Q. What are the receive and sending limits of Office365?
+
+Q. The sync fails with many "Trying command when NOT connected!".
+   What can I do?
+
+Q. How to see or migrate public folders on Office365?
+R. https://docs.microsoft.com/en-us/exchange/collaboration/public-folders/migrate-to-exchange-online?view=exchserver-2019
+   https://www.exchangesavvy.com/moving-your-public-folders-to-office-365-what-you-need-to-know/
+
+Q. Office365 refuses to create the folder named "Files" with the error
+   "NO Folder name is reserved". What happens?
+
+Q. Office365 users complain that a folder named "Files" contains 
+   messages with no sender.
+
+Q. From XXX to Office365, read receipts are all resent again after sync.
+   Even for old messages. How can I fix that?
+
+Q. DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error
+   DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error   
+
+Q. From XXX Office365 I get this error message sometimes: 
+   "BAD Command Argument Error 11". What does it mean?
+
+Q. From XXX to Office365 the flag Flagged does not seem to be well synced. 
+   What can I do?
+
+Q. How to migrate from or to Office 365 with an admin/authuser account?
+
+Q. Couldn't create folder [trash] "Mailbox already exists".
+
+
+Now the questions again with their answers.
+
+======================================================================
+Q. Can I use imapsync to transfer from or to Office365 accounts?
+
+R0. Yes. But IMAP access to Office365 account is not always allowed 
+by default so it has to be allowed in the server configuration part.
+   
+https://docs.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/pop3-and-imap4/enable-or-disable-pop3-or-imap4-access
+https://docs.microsoft.com/en-us/exchange/troubleshoot/configure-mailboxes/pop3-imap-owa-activesync-office-365
+
+R1. Imapsync doesn't support OAUTH2 authentication for Office365 
+accounts yet. That's not totally true, if you have an access token
+to access your mailbox then you can authenticate with imapsync this
+way:
+
+  imapsync ... --oauthaccesstoken1 tokenfile
+
+where "tokenfile" is a file containing the access token.
+
+R2. Enable double-step authentication and configure it but after use 
+an "app password" with imapsync. Details:
+
+a) Go to 
+https://account.microsoft.com/security
+
+b) Click on "Advanced Security Options"
+Turn "Two-step verification" on. Follow the steps and finish".
+
+c) Then now "App passwords" is available.
+Click on "Create a new app password". 
+Use this password to authenticate with imapsync.
+
+d) Delete this app password when the job with imapsync is finished.
+
+R3. Also, check a license is assigned to that account in Office365.
+
+R4. From Dave Pusey 
+https://github.com/imapsync/imapsync/issues/317#issuecomment-1027776418
+
+I quote Dave nearly verbatim:
+
+"I had created an app password, and security defaults are already off.
+
+I have now figured out the issue.
+
+Despite the MS365 and EXO admin centers showing that IMAP and Basic
+Auth were all enabled, it turns out that in Oct 2021, Microsoft began
+disabling basic auth for all tenants that had have never used it by
+that point. There was an item in my Message Center from that date
+saying my tenant was being done.
+
+You can re-enable it for specific protocols (IMAP in this case) using
+the diagnostic process detailed at
+https://techcommunity.microsoft.com/t5/exchange-team-blog/basic-authentication-and-exchange-online-september-2021-update/bc-p/2782230
+(see the section "Limited Opt Out")
+
+R5. Microsoft introduced something called "security defaults" which is enabled
+by default for new tenants. One of the rules blocks IMAP access as of imapsync.
+
+The funny thing is that you can't disable a single rule of this 
+security package without buying additional licenses.
+Switching the whole thing off allows the IMAP login.
+
+Also, disable double-step authentication on the Azure/Active Directory portal.
+See here:
+https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/concept-fundamentals-security-defaults
+Thanks to Stephan Buhre for this R5 answer.
+
+R6. Are there special characters in the password?
+    https://imapsync.lamiral.info/FAQ.d/FAQ.Authentication_failure.txt
+    https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Windows.txt
+    https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt
+    https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Mac.txt
+
+R7. Triple check the hostname then. Try all of these:
+ * outlook.office365.com
+ * imap-mail.outlook.com
+ * imap.outlook.com
+
+======================================================================
+Q. Can imapsync work for users that are administrators for 
+an Office 365 domain? (quick answer: no)
+
+R. It doesn't seem possible to use imap for administrators, so imapsync
+won't be able to work for an administrator's mailbox.
+See:
+https://github.com/imapsync/imapsync/issues/310#issuecomment-1002396218
+https://exhaust.lewiscollard.com/post/146866104/office365-to-migadu-migration/
+
+Solution: as explained in the article above, use Davmail as a proxy to 
+access this mailbox. http://davmail.sourceforge.net/
+
+Thanks to Lewis Collard for this report and solution.
+
+======================================================================
+Q. Does imapsync support OAUTH2 authentication for Office365 accounts?
+
+R. Yes but partially. Imapsync won't help you getting an access token
+but if you have one then you can use it with imapsync this way:
+
+Office365 as source:
+
+  imapsync ... --oauthaccesstoken1 tokenfile1
+
+Office365 as destination:
+
+  imapsync ... --oauthaccesstoken2 tokenfile2
+
+======================================================================
+Q. How to sync from Office365 to XXX?
+
+R0. IMAP is not enabled by default on Office365, see how to enable it:
+https://docs.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/pop3-and-imap4/enable-or-disable-pop3-or-imap4-access
+https://docs.microsoft.com/en-us/exchange/troubleshoot/configure-mailboxes/pop3-imap-owa-activesync-office-365
+
+If IMAP cannot be enabled or if it doesn't work well, you can try DavMail 
+(http://davmail.sourceforge.net/) which acts as an IMAP gateway 
+through Outlook Web Access.
+
+R. On Windows, use:
+
+  imapsync.exe ... --office1
+
+On Unix, use:
+
+  imapsync ... --office1
+
+Option --office1 is like (release 1.970 or higher):
+  
+  imapsync ... --host1 outlook.office365.com \
+               --ssl1 \
+               --exclude "^Files$" 
+  
+======================================================================
+Q. How to sync from XXX to Office365
+
+R0. IMAP is not enabled by default on Office365, see how to enable it:
+https://docs.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/pop3-and-imap4/enable-or-disable-pop3-or-imap4-access
+https://docs.microsoft.com/en-us/exchange/troubleshoot/configure-mailboxes/pop3-imap-owa-activesync-office-365
+
+If IMAP cannot be enabled or if it doesn't work well, you can try DavMail 
+(http://davmail.sourceforge.net/) which acts as an IMAP gateway 
+through Outlook Web Access.
+
+R. Here is a command-line summary that solves most encountered issues when 
+   migrating to Office365.
+
+    imapsync ... --office2
+
+which is equivalent to (in imapsync release 1.870 or higher):
+
+  imapsync ... \
+    --host2 outlook.office365.com \
+    --ssl2 \
+    --maxsize 45000000 \
+    --maxmessagespersecond 4 \
+    --disarmreadreceipts \
+    --regexmess "s,(.{10239}),$1\r\n,g" \
+    --f1f2 "Files=Files_renamed_by_imapsync"
+
+
+On Linux, you can also try the "reformime" command 
+that can be used like:
+
+  imapsync ... --maxlinelengthcmd "reformime -r7"
+
+To get reformime, install the "maildrop" package.
+
+On Linux again, there is a good Python script in the tarball that can 
+fix several things that Exchange or O365 have issues with.
+
+Use it like this:
+
+  ./imapsync ... --pipemess W/tools/fix_email_for_exchange.py
+
+It often does some miracles on messages.
+
+======================================================================
+Q. For Office365 I have double and triple-checked the username and 
+password spelling but I still get a "LOGIN failed". Any clue?
+
+R. Go to "Q. Can I use imapsync to transfer from or to Office365 accounts?"
+
+======================================================================
+Q. How can I access an Office365 shared mailbox?
+
+R. First, create a shared mailbox, for example shared@example.com. 
+Then give full permissions to a licensed account user@example.com.
+Now with imapsync, use the licensed user login with the syntax: 
+
+  user@example.com\shared@example.com
+
+and the password for user@example.com
+
+Caveat: Character \ is a special character, so
+use double-quotes around the user login, like:
+
+  imapsync ... --user1 "user@example.com\shared@example.com"
+
+Sources:
+https://adam-hand.com/2017/07/25/connect-a-shared-mailbox-from-o365-to-outlook-via-imap/
+https://www.arclab.com/en/kb/email/imap-settings-shared-mailbox-office-exchange-online.html
+https://social.technet.microsoft.com/Forums/en-US/336e02ee-6767-4810-90a0-1352bd7cc9e9/office-365-how-to-access-a-shared-mailbox-using-imap-client?forum=onlineservicesexchange
+
+======================================================================
+Q. How can I use a shared account as a backup account for several
+mailboxes and so avoid spending too many dollars in backup accounts?
+
+R. Use the option --subfolder2 backup_foo
+
+  imapsync ... --user2 sharedloginsyntax --subfolder2 backup_foo
+
+To restore, do the reverse with:
+
+  imapsync ... --user1 sharedloginsyntax --subfolder1 backup_foo
+
+
+======================================================================
+Q. Office365 fails with "User is authenticated but not connected".
+
+R1. "The message User is authenticated but not connected is due to a 
+   bug in the Office365 server's IMAP implementation. If the client 
+   presents a valid user name but an invalid password, the server 
+   accepts the login, but subsequent commands fail with the 
+   aforementioned error message." Source:
+https://unix.stackexchange.com/questions/164823/user-is-authenticated-but-not-connected-after-changing-my-exchange-password
+Thanks to James Abbottsmith for this link and explanation at
+https://github.com/imapsync/imapsync/issues/32#issuecomment-153561647
+
+R2. Miguel Alameda reported understanding and solving this issue 
+    like this, the context was admin/authuser:
+    "The admin user had not permission in the target mailbox."
+
+======================================================================
+Q. I see "NO Maximum size of appendable message has been exceeded"
+   What can I do with that?
+
+R. Office365 supports send/receive max message sizes of up to 150MB 
+   but you need to make changes in your tenant(s) to support it.
+
+The following PowerShell command will increase the message sizes that 
+can be sent/received.  The trick in getting IMAPSync to work is to 
+apply these settings to the accounts performing the migration,
+NOT the accounts associated with the target mailbox (assuming you're
+using service accounts to perform transfers on behalf of users).
+
+  Set-mailbox -Identity $UPN -MaxReceiveSize 150mb -MaxSendSize 150mb
+
+e.g. 
+
+  Set-mailbox -Identity "migrationaccount@testtenant.onmicrosoft.com" -MaxReceiveSize 150mb -MaxSendSize 150mb
+
+We're transferring data between Office 365 tenants so we set these 
+values on the migration accounts in the source and target tenants.
+
+Thanks to Sean McDougall, Ian Thomas & Matt Wilks from Toronto 
+for this FAQ item.
+
+======================================================================
+Q. Every single mail synced to exchange online owns the 
+   category $MDNSent after migration. How can I avoid this?
+
+R. To remove the flag $MDNSent from all messages, use:
+
+  imapsync ... --regexflag "s/\$MDNSent//g"
+
+See also the document
+https://imapsync.lamiral.info/FAQ.d/FAQ.Flags.txt
+
+======================================================================
+Q. The imap connection to Office365 is not working very efficiently,
+   is there a solution to fix that?
+
+R. Yes. Try DavMail
+   http://davmail.sourceforge.net/
+   I don't use it myself but a user, Yannick Palanque, reported
+   great results using it.
+
+======================================================================
+Q. The sync fails with many "Trying command when NOT connected!".
+   What can I do?
+
+R. The --debugimap option can show you more details, especially 
+   messages like this one:
+   "BAD Request is throttled. Suggested Backoff Time: 178755 milliseconds"
+   In that case, see the next faq item and its fixes.
+
+======================================================================
+Q. Office365 throttles the sync and says: 
+   "Request is throttled. Suggested Backoff Time: 299961 milliseconds".
+   What can I do with that?
+
+R. Office365 has throttle mechanisms to limit any huge usage.
+   Sometimes imapsync transfers are too stressful for servers.
+   The following message 
+   "Request is throttled. Suggested Backoff Time: 299961 milliseconds"
+   comes from the imap Office365 server, imapsync just reports it
+   before being disconnected from it.
+
+   To solve the throttles issues from 0365, there are two solutions
+   at least:
+
+R1. Call Microsoft Office365 and ask them to remove the limits on your 
+  mailboxes. That's not a joke, they do it for 90 days usually, 
+  sometimes only after you reach the second technician you call, 
+  the first one usually be not enough competent to understand 
+  what you're talking about (I would be glad to remove this bad
+  fact).
+
+R2. Play with options --maxbytespersecond or --maxmessagespersecond
+    or --exitwhenover
+
+  imapsync ... --maxbytespersecond 100_000
+
+  imapsync ... --maxmessagespersecond 2
+
+  imapsync ... --exitwhenover 1_000_000_000
+
+I don't know the upper value that avoids the default throttling from 0365
+and I guess it changes over time.
+
+======================================================================
+Q. What are the receive and sending limits of Office365?
+
+R1. 3600 messages per hour (22 July 2021). Maybe 1200 as imapsync can 
+   be viewed as a single sender. Tell me what you experienced, 
+   I'll update this point.
+
+https://docs.microsoft.com/en-us/office365/servicedescriptions/exchange-online-service-description/exchange-online-limits#receiving-and-sending-limits
+
+In that case, imapsync can adapt to this with (1 message/second = 3600 messages per hour)
+
+    imapsync ... --maxmessagespersecond 1
+or
+    imapsync ... --maxmessagespersecond 0.33
+
+R2. I also found "Microsoft theoretically allows for about 300MB of
+throughput per user per hour."  at
+https://www.systools.in/blog/microsoft-office-365-throttling-policy/
+
+In that case, imapsync can adapt to this with (83333 bytes/second = 300 MBytes/hour)
+
+    imapsync ... --maxbytespersecond 83333
+
+======================================================================
+Q. Office365 refuses to create the folder named "Files" with the error
+   "NO Folder name is reserved". What happens?
+
+R. The folder Files is a standard folder in Office365.
+   It should not be synced in IMAP. See the next question.
+
+======================================================================
+Q. Office365 users complain that a folder named "Files" contains 
+   messages with no sender.
+
+R0. To fix this, add --exclude Files
+
+  imapsync ... --exclude Files
+
+  If you use --office1 then imapsync will add this exclusion 
+  automatically like using  the option:
+  
+  imapsync ... --exclude "^Files$" 
+
+  If you use --office2 then imapsync will add a renaming of any
+  "Files" folder on host1, like using the option:
+  
+  imapsync ... --f1f2 "Files=Files_renamed_by_imapsync"
+
+  The host2 account ends up with a folder named 
+  "Files_renamed_by_imapsync", but no complaining.
+
+R1. This folder "Files" seems to be a standard folder in Exchange Online,
+    but it is not. This folder contains all attachments in every email
+    that is in the mailbox but without any headers.
+    
+    This causes some confusion for users as these appear in their search
+    results as duplicate lines but without the sender details or even
+    the message body.
+
+   This folder seems to be usually hidden so IMAP clients can’t see it, 
+   but for some reason sometimes it becomes visible.
+
+R2. It looks to be a common problem with Exchange Online. 
+    I’m not sure what causes the folder to appear.
+
+   More info here:
+https://answers.microsoft.com/en-us/msoffice/forum/msoffice_o365admin-mso_exchon-mso_o365b/exclude-the-exchange-online-system-folder-called/2adbdf84-db4a-4c7f-ac29-738757980a0d
+https://answers.microsoft.com/en-us/msoffice/forum/msoffice_o365admin-mso_dep365-mso_o365b/no-sender-emails-in-files-folder/534bae8f-a7d7-4f5f-8ed7-5bad0d5fa23f
+
+  (This question/answer is taken quasi verbatim from Perttu Aaltonen)
+
+======================================================================
+Q. From XXX to Office365, read receipts are all resent again after sync.
+   Even for old messages. How can I fix that?
+
+R. Imapsync can remove the header containing this read-receipt request.
+
+On Unix or Windows use:
+
+  imapsync ... --disarmreadreceipts
+
+Since read receipts should be sent for unseen messages that will go 
+to a seen state after the migration, you could be strict and apply 
+the regex only to seen messages. 
+Selecting seen message can be done with:
+
+  imapsync ... --search1 "SEEN" --disarmreadreceipts
+
+A second run has to be run without the --disarmreadreceipts
+for unseen messages:
+
+  imapsync ... --search1 "UNSEEN" 
+
+If fact --disarmreadreceipts is just an option equivalent to:
+
+  --regexmess 's{\A(.*?(?! ^$))^Disposition-Notification-To:(.*?)$}{$1X-Disposition-Notification-To:$2}igxms' 
+
+That regex changes the header Disposition-Notification-To. It prefixes
+it with an X- so that it becomes inactive.
+Disposition-Notification-To: blabla
+becomes
+X-Disposition-Notification-To: blabla
+
+Thanks to David Karnowski for pointing and solving this issue.
+
+======================================================================
+Q. DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error
+   DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error   
+
+R1. "SSL read or write error" happens sometimes, it isn't related to 
+    imapsync directly but to the ssl underlying library when communicating
+    with Exchange in TLS/SSL encrypted mode. 
+    Next runs should put the sync further, so rerun the syncs 
+    until it is well completed.
+
+R2. Another solution is to remove --tls or --ssl options for Exchange
+    and accept clear text syncs.
+
+R3. See also the FAQ FAQ.SSL_errors.txt
+    https://imapsync.lamiral.info/FAQ.d/FAQ.SSL_errors.txt
+
+======================================================================
+Q. From XXX Office365 I get this error message sometimes: 
+   "BAD Command Argument Error 11". What does it mean?
+
+R. This error message comes from the Office365 IMAP server when it
+encounters any problem. Most of the time it is one of the following:
+
+  * Some messages are bigger than the size limit. 
+    45 MB by default on Office365.
+    I don't know if it can be upped by configuration for Office365.
+    If you can't fix this limit on Office365 then use the option
+    --maxsize 45000000 for 45 MB to tell imapsync to skip those messages.
+
+  imapsync ... --maxsize 45000000 # 45 MB for Office365
+  
+
+  * Quota reached. The whole account is full.
+    It can be upped by configuration.
+
+  * You use --synclabels --resynclabels from a previous command line
+    related to syncing from Gmail to Gmail. Remove them.
+
+  * Some messages have some lines too long. Use option --maxlinelength
+    to skip messages whose max line length is over some bytes.
+    --maxlinelength 1000 is an RFC2822 must but most servers support
+    higher values. Office365 supports 10500 characters line length:
+
+  imapsync ... --maxlinelength 10500
+
+  In case you prefer fixing messages with long lines the hard way,
+  instead of skipping them with --maxlinelength 10500, just use:
+
+  On Windows
+  imapsync ... --regexmess "s,(.{10500}),$1\r\n,g"
+
+  On Unix
+  imapsync ... --regexmess 's,(.{10500}),$1\r\n,g'
+  
+Have also in mind that Office365 closes the connection after 10 errors
+encountered so you might also see "BYE Connection closed" errors from
+Office365, which means Office365 leaves the session and says goodbye,
+come back later. Redo some sync then.
+
+On Linux, there is a good Python script in the tarball that can 
+fix several things that Office365 has issues with.
+
+Use it like this:
+
+  ./imapsync ... --pipemess W/tools/fix_email_for_exchange.py
+
+It often does some miracles on messages.
+It's called fix_email_for_exchange because Office365 is Exchange,
+or at least started to be an Exchange server with the same issues.
+
+======================================================================
+Q. From XXX to Office365 the flag Flagged does not seem to be well synced. 
+   What can I do?
+
+R. Use the following trick. Run imapsync twice, one with --regexflag
+   and one without, like this:
+
+1) imapsync ... --regexflag "s/\\Flagged//g"
+2) imapsync ...
+
+You can add --debugflags if you want to see what imapsync gets and 
+does in detail with flags.
+
+The magic of this trick is on ignoring the \Flagged flag on the first 
+sync and setting it on the second sync, with STORE instead of APPEND.
+This Office365 bug seems that Office365 gets and sets well the Flagged 
+flag with APPEND in IMAP but then it forgets it with other protocols;
+With STORE it sets and gets the "\Flagged" flag everywhere.
+
+Thanks to Dave Murray and Simon Savva for reporting and solving 
+this issue.
+
+======================================================================
+Q. How to migrate from or to Office 365 with an admin/authuser account?
+
+Note from Yago Torres Fernandez:
+(a working command using admin/authuser on host2 Office 365)
+
+  imapsync ... --authuser2 user_admin@domain.com --user2 user_to_be_migrated@domain.com  ^
+               --password2 XXXX --ssl2
+
+but previously in Office365, you must do something like the following, using Powershell:
+
+  Add-MailboxPermission -identity user_to_be_migrated@domain.com -user user_admin@domain.com -accessrights fullaccess -inheritancetype all
+
+
+Note from Betsy Lawlor: You can use global modern authentication with two factor 
+on Exchange Online (M365) but you must have "AllowBasicAuthImap" on
+the admin account you are using to migrate the mail.
+
+Note from Guido (5 April 2022):The way I fixed it was by turning off security defaults 
+https://docs.microsoft.com/nl-nl/azure/active-directory/fundamentals/concept-fundamentals-security-defaults. 
+You still need to check IMAP access on an account-basis though.
+
+Remark: PLAIN authentication is the only way to go with --authuser1 for now.
+So don't use --authmech1 SOMETHING with --authuser1 admin_user,
+it will not work.
+Same behavior with the --authuser2 option.
+
+Note from Rafael Alvarez Ballesteros:
+When you get an Office365 license you will receive an admin user to handle
+your licenses and products like admin@yourcompanyname.onmicrosoft.com.
+This account is the administrator account; some weeks or months ago Microsoft 
+has decided you need to use two-factor authentication by default, 
+so if two factors authentication is enabled you will no be able to sync 
+the mail (it will not connect to host2).
+OWA (I think this is the two-factor authentication) needs to be disabled 
+globally and enabled individually on the users you want to.
+One account can have the right to access other mailboxes no matter if admin or 
+any other account but needs to have OWA disabled to be able to connect 
+to the office365 server.
+
+Note from Martin Paulucci:
+I had to remove the domain part for the user
+but not for the admin.  Example:
+
+  imapsync ... --authuser2 user_admin@domain.com --user2 user_to_be_migrated
+
+See also:
+http://linux-france.tk/prj/imapsync_list/msg02203.html
+Subject: RE: [imapsync] Office 365 - 'Master User'?
+Date:    Mon, 1 Jun 2015 17:53:54 +0000
+
+
+======================================================================
+Q. Couldn't create folder [trash] "Mailbox already exists".
+
+R. Some servers take care of character cases in folder names, 
+   some servers do not, like Exchange. Since not respecting the case
+   can merge two different folders into one then imapsync respects the case. 
+
+   For example, if a host1 server has a folder name called "trash"
+   and the host2 server already has a folder "Trash" or "TRASH"
+   then imapsync will try to create the folder "trash" on host2
+   because trash and Trash are different strings. But if host2
+   does not respect character case it will consider folder "trash"
+   already exists and will say it. That's the error message reported
+   by imapsync: "Mailbox already exists". This message comes from the server.
+
+   The folder creation fails but messages are well transferred in
+   so take a look at this warning, understand why it happens
+   and it should be fine most of the time.
+
+   To avoid this warning use --regextrans2 to map the folder names
+
+   imapsync ...  --regextrans2 "s/^trash$/Trash/"
+
+   If there are two folders Trash and trash on host1 then both
+   will be merged into only one Trash folder on host2. 
+   In case of the option --delete2 is used the regextans2 above becomes
+   mandatory, otherwise imapsync will sync messages from the
+   first Trash and then delete them when syncing trash.
+
+   If you want to avoid merging folders that are considered different 
+   on host1 but are considered the same on destination host2 because of
+   case sensitivities and insensitivities, use  --nomixfolders
+
+======================================================================
+======================================================================

FAQ.d/FAQ.Old_Style_Web_Design.txt

@@ -0,0 +1,66 @@
+#!/bin/cat
+$Id: FAQ.Old_Style_Web_Design.txt,v 1.9 2020/12/02 21:38:13 gilles Exp gilles $
+
+This document is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Old_Style_Web_Design.txt
+
+=======================================================================
+              Imapsync and its ugly website style
+=======================================================================
+
+=======================================================================
+Q. Why does the imapsync website look so old fashioned?
+
+R. Once in a while I receive messages from people complaining about 
+the website style, not that much about the content by the way,
+as people don't read.
+
+I know the imapsync website has a grandmother style,
+I'm not offended by complaints about it.
+
+Here are several reasons it stays like that for now:
+
+* I like to do things myself
+
+* I am lazy
+
+* I like static web.
+  Although you could argue that I do use some dynamic things, like SSI. 
+  That's true, I use Server Side Include, aka SSI, denoted by the ".shtml" 
+  extension of some files, instead of the classical ".html" extension.
+  It's because I like dynamic web when the dynamic brings something that
+  can't be done easily with static files.
+  
+
+* I'm proud to be w3c compliant
+  https://validator.w3.org/check?uri=https%3A%2F%2Fimapsync.lamiral.info
+  Very very few sites are w3c compliant, it's because nearly all CMS frameworks aren't.
+  Only true crazy people are w3c compliant seekers. I'm one of them.
+  If you know a CMS w3c compliant, tell me, it will be such a crazy news.
+  If you know any web site w3c compliant, tell me, it will be crazy news as well.
+  Update: https://contao.org/ is a CMS and it is w3c compliant! This is astonishing!
+  (Thanks to Markus Rupprecht for this crazy input!)
+  
+* I prefer to spend more time on the product itself than the showcase
+
+* The product is a command line, a fancy website to sell a command line
+  makes more disappointed buyers
+
+I agree I could do some sort of A/B experiment and see whether during
+a month I sell more imapsync on a brand new style B site than
+the current A one.
+
+
+Besides, don't misunderstand buyers. The surprising thing about complainers
+is that all are web designers, none of them is a user not in the webdesign branch.
+
+I even received some phone call from a buyer saying: "I saw your website 
+and I told to myself: this can't be a fake one, the product must be a good one".
+
+Another folk like me:
+http://keyhut.com/pos.htm
+(well, not w3c compliant, he is not that mad)
+
+So maybe one day...
+

FAQ.d/FAQ.OnlineUI.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.OnlineUI.txt,v 1.19 2019/01/28 10:59:16 gilles Exp gilles $
+$Id: FAQ.OnlineUI.txt,v 1.36 2022/09/14 11:18:05 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -7,128 +7,294 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.OnlineUI.txt
 
 =====================================================================
    Imapsync tips about the online visual user interfaces
-   https://i005.lamiral.info/X/
    https://imapsync.lamiral.info/X/
 =====================================================================
 
 Questions answered in this FAQ are:
 
+Q. Can I launch several imap synchronizations on the visual 
+   user interface /X?
+
 Q. How secure is the online visual user interface /X?
 
-Q. Will I have any issues with browser timing out? What happens
-   if the browser connection is closed for whatever reason?
+Q. Does the online service store any sensitive information 
+   like my passwords?
 
+Q. I want to switch from the visual interface /X to the 
+   imapsync command line or to the script examples
+   https://imapsync.lamiral.info/#DOC_BASIC
+   What should I know?
+
+Q. Shall I have issues when the browser times out?
+   What happens if the browser connection is closed for whatever reason?
+
+Q. Shall I have issues when the webserver times out? What happens
+   if the web server closes the connection for whatever reason?
+
+Q. The sync stalls at the beginning, just after a line like:
+   "Host1: xxx says it has CAPABILITY for AUTHENTICATE LOGIN"
+   What is the problem?
+
+Q. The synchronization fails with the error message like:
+   Err 1/1: Host1 failure: Error login on [10.1.161.155] with user [webmaster@truc.com] auth [LOGIN]: 2 NO [ALERT] LOGIN DENIED -- COUNTRY IS BLACKLISTED
 
 Now the questions again with their answers.
 
 =====================================================================
-Q. How secure is the online visual user interface /X?
+Q. Can I launch several imap synchronizations on the visual 
+   user interface /X?
 
-R0. Well, I don't know if asking the provider whether his online 
-   service is secure or not would be of any interest. 
-   Let's do it anyway, you'll be the judge.
-
-R1. Some figures
-
-Date of this report: 26 December 2018.
-
-The online imapsync service /X started 9 January 2017 
-(716 days of service).
-
-In average, /X has 20 users per day lunching in mean 6 
-different migrations, from one launch to many (hundreds).
-
-The total volume /X transferred is around 45 TiB in more 
-than 111 thousands email imap migrations, 
-157 millions email messages.
-
-R2. Pros & Cons
-
-The online imapsync service /X runs on https only, with a 
-letsencrypt certificate, a certificate overall rated "A+" at
-https://www.ssllabs.com/ssltest/analyze.html?d=i005.lamiral.info
-
-Because of the https usage, what the users enter in their browser,
-the imap logins and passwords, can't be eavesdropped on the network.
-
-Imapsync itself takes care about encryption for the imap sessions, 
-if possible: It tries SSL first on port 993, then TLS on port 143 
-if the servers announces TLS, then no encryption at all. 
-What is done with the source imap server host1 is independent 
-of what is done with the destination imap server host2.
-
-At the date of 26 December 2018, there is no security problem 
-detected or reported to me (Gilles LAMIRAL), so far.
-Feel free to attack the service and feel free to report any 
-hole encountered. Have in mind I can watch what you try 
-from the server side.
-
-As the owner of the service, it could have been 48 000 pairs of 
-credentials collected and nearly 17 terabytes of email messages. 
-I haven't kept them but I can't prove I haven't. It's just trust, 
-like nearly every online service in the universe.
-
-The imap server certificates are not checked (by default) 
-because too many  imap servers are crappy configured regarding 
-certificates.
-
-This default behavior is chosen like this because users of /X 
-want their emails transferred, instead of being not transferred 
-because of an incompetent imap server sysadmin.
-
-I admint that this part, checking imap ssl/tls certificates, 
-could be improved from my side by including well known 
-certificates directly in imapsync. 
-
-If the imap servers don't honor ssl nor tls, then logins, passwords 
-and everything will go clear text during the imap transfers. 
-That's not good at all but what "comforts" me is that if the 
-imap servers do only clear text transfers, then it's also true 
-for all imap sessions the accounts' owner encounters, 
-imapsync is just one of them.
-
-Last point, who could be sure that no cracker cracked the online 
-hosts and that he isn't currently sniffing the credentials?
-
-No one, I'm not sure myself, even if I do take  care of that 
-possibility. So changing the imap accounts passwords after 
-a sync is a safe and recommended practice!
+A. Yes. Open several tabs/windows on your browser and fill each one
+   with different credentials.
 
 =====================================================================
-Q. Will I have any issues with browser timing out? What happens
-   if the browser connection is closed for whatever reason?
+Q. How secure is the online visual user interface /X?
 
-R. It stops the imapsync process, ie, the sync is ended right away.
+A0. Well, I don't know if asking the provider whether his online
+service is secure or not would be of any interest.
+Let's do it anyway, you'll be the judge.
+
+A1. Some figures
+
+Date of this report: 24 December 2021. Happy Christmas!
+
+The online imapsync service /X started 9 January 2017,
+5 years ago (1810 days of service).
+
+On average, /X has 51 users per day, each user launches on average
+6 account migrations, from just 1 launch to many (hundreds).
+
+The total volume /X transferred so far is around 240 TiB coming 
+from nearly 520 000 imap account migrations and 810 million email messages.
+
+A2. Pros & Cons
+
+The online imapsync service /X runs on HTTPS only, with a letsencrypt
+certificate, an up to date certificate overall rated "A+" at
+https://www.ssllabs.com/ssltest/analyze.html?d=imapsync.lamiral.info
+
+Because of the HTTPS usage, what the users enter in their browser, the
+imap logins and passwords, can't be eavesdropped on the network.
+
+Imapsync itself takes care of encryption for the imap sessions, if
+possible. First, imapsync tries to use SSL on port 993, then TLS on
+port 143 if the servers announce TLS, then no encryption at all.
+Concerning encryption, what is done with the source imap server host1
+is independent of what is done with the destination imap server host2.
+
+On the date of 24 December 2021, there is no security problem detected
+or reported to me (Gilles LAMIRAL), so far.
+
+Concerning the Log4j threats, the imapsync server doesn't use Log4j at
+all, so it should be safe about this issue.
+
+Feel free to attack the service and feel free to report any hole
+encountered. Have in mind that I can watch what you do from the
+server-side, and then take measures if the service suffers from your
+acts. Drop me a note before will be fair play and I will let you act
+as harsh as you can.
+
+As the owner of the service, it could have been 520 000 pairs of
+credentials collected and nearly 240 terabytes of email messages.
+That's massive. I haven't kept them but I can't prove I haven't. It's
+just trust, like nearly every online service in the universe.
+
+Concerning imapsync transfers, the imap server certificates are not
+checked for authenticity by default because too many imap servers are
+crappy configured regarding their certified certificates.
+
+This default behavior is chosen like this because users of /X
+want their emails transferred, instead of being not transferred
+because of an incompetent imap server sysadmin.
+
+I admit that this part, checking imap ssl/tls certificates,
+could be improved from my side by including well known
+certificates directly in imapsync.
+
+If an imap server doesn't honor ssl nor tls, then logins, passwords
+and everything will go clear text during the imap transfers.  That's
+not good at all! But what "comforts" me is that if an imap server does
+only clear text transfers, then it's also true for all the imap
+sessions the account owner encounters, imapsync is just one of them;
+the imap servers are then always unsecured for any imap software
+client.
+
+Last point, who could be sure that no cracker cracked the online hosts
+and that he isn't currently sniffing the credentials?
+
+No one! I'm not sure myself, even if I do take care of that
+possibility. So changing the imap accounts passwords after
+the sync is a safe and recommended practice! You can even
+change the password just after imapsync has started its job, 
+just after a successful login. This way you know the previously 
+given password is no longer working just after imapsync has
+started its job.
+
+=====================================================================
+Q. Does the online service store any sensitive information 
+   like my passwords?
+
+No.
+
+The online passwords are kept by your browser. The goal is that the
+next time you run a sync from your browser, all the parameters are
+already there without having to re-enter them.  The save is done when
+you click on the "Sync or resync" green button. So, if you don't want
+your browser to keep your password, enter a wrong one or empty the
+password field and click on the "Sync or resync" green button, the
+wrong password will be saved in your browser and no sync will be done,
+since a correct login is mandatory to do anything on your mailboxes.
+
+=====================================================================
+Q. I want to switch from the visual interface /X to the 
+   imapsync command line or to the script examples
+   https://imapsync.lamiral.info/#DOC_BASIC
+   What should I know?
+
+A. Let's do some ascii art. 
+   The visual interface looks roughly like this, for the textfields input part:
+
++------------------------------------------+------------------------------------------+
+|          IMAP source Mailbox             |        IMAP destination Mailbox          |
+|                                          |                                          |
+| Login (usually an email address)         | Login (usually an email address)         |
+|                                          |                                          |
+|    test1                                 |    test2                                 |
+|                                          |                                          |
+| Password                                 | Password                                 |
+|                                          |                                          |
+|    secret1                               |    secret2                               |
+|                                          |                                          |
+| IMAP Server hostname (or its IP address) | IMAP Server hostname (or its IP address) |
+|                                          |                                          |
+|     test1.lamiral.info                   |     test2.lamiral.info                   |
+|                                          |                                          |
++------------------------------------------+------------------------------------------+
+
+Notice the 6 examples values I put in the previous "picture":
+
+* test1
+* secret1
+* test1.lamiral.info
+
+* test2
+* secret2
+* test2.lamiral.info
+
+
+The corresponding imapsync command line on Windows is:
+
+imapsync.exe  --host1 "test1.lamiral.info"  --user1 "test1" --password1  "secret1"  ^
+              --host2 "test2.lamiral.info"  --user2 "test2" --password2  "secret2"
+
+The order of the  parameters is whatever you want as long as you respect each pair 
+like: --optionname "value"
+
+There is no need to type this in a command prompt window, just use the example script 
+https://imapsync.lamiral.info/examples/imapsync_example.bat
+
+
+The corresponding imapsync command line on Linux is:
+
+imapsync  --host1 "test1.lamiral.info"  --user1 "test1" --password1  "secret1"  \
+          --host2 "test2.lamiral.info"  --user2 "test2" --password2  "secret2"
+
+There is no need to type this in a command prompt window, just use the example script 
+https://imapsync.lamiral.info/examples/imapsync_example.sh
+
+=====================================================================
+Q. Shall I have issues with the browser timing out?
+   What happens if the browser connection is closed for whatever reason?
+
+A. A browser connection closed closes also the imapsync process, 
+   ie, the sync is ended right away.
 
 Further comments on this behavior.
 
-When using the /X interface there are three connections. 
-One connection is the Browser/WebServer connection, 
-the two others connections are the WebServer/ImapServers 
+When using the /X interface there are three connections.
+One connection is the Browser/WebServer connection,
+the two other connections are the WebServer/ImapServers
 connections (imapsync stuff).
 
-If the Browser/WebServer connection is timeout or ended, 
-the imapsync sync is also ended immediately by the remote 
-Apache https server. Technically, Apache sends a TERM signal,
-to the imapsync process then wait some seconds before 
-sending a KILL signal if it is still alive.
+If the Browser/WebServer connection timeouts or ends,
+the imapsync sync is also ended immediately by the remote
+Apache HTTPS server. Technically, Apache sends a TERM signal
+to the imapsync process, then wait some seconds before
+sending a KILL signal if the imapsync process is still alive.
 
-You can relaunch a sync again with "Sync!" button, at any time.
-If the "Sync!" button is gray/inactive then just reload 
+You can relaunch the sync with the "Sync!" button, at any time.
+If the "Sync!" button is gray/inactive then just reload
 the page (F5 or similar), and reenter the credentials.
 
-If the interface tells you that a sync is already going on,
-it may be that a sync is running from another browser or place.
+If the interface tells you that the sync is already going on,
+it may be that the sync is running from another browser or place.
 You can stop this sync with the "Abort!" button from any /X
 tab/window, even from another browser or place. To be able
-to abort with success, you have to give the same account 
+to abort with success, you have to give the same account
 parameters, same credentials, or imapsync will ignore the demand.
 
-In other words, you can try safely to launch several parallel 
-runs between the same mailboxes. Open a new tab/windows with /X, 
-and start a same sync, it's safe, the /X will say, if any, that 
-there is already a current sync running.
+In other words, you can try safely to launch several parallel
+runs between the same mailboxes. Open a new tab/windows with /X,
+and start the same sync. It's safe, the /X page will say that
+there is already one sync running and it will present
+the logfile running the sync like a "tail -f" command (isn't that magic?).
 
+=====================================================================
+Q. Shall I have issues when the webserver times out? What happens
+   if the web server closes the connection for whatever reason?
+
+A. If the webserver closes the connection then usually it also 
+   kills the imapsync process and the imap connections as well.
+
+The current webserver timeout at 
+https://imapsync.lamiral.info/X/
+is 3600 secondes, one hour.
+
+=====================================================================
+Q. The sync stalls at the beginning, just after a line like:
+   "Host1: xxx says it has CAPABILITY for AUTHENTICATE LOGIN"
+   What is the problem?
+
+A. I've seen this issue on /X with the imap server
+   CommuniGate Pro IMAP Server 6.0.11
+
+The issue looks related to special characters in the password.
+Solution for now: change the password, keep only standard (ASCII) 
+alphanumeric characters ABC-YZ abc-yz 012-89.
+
+=====================================================================
+Q. The synchronization fails with the error message like:
+   Err 1/1: Host1 failure: Error login on [10.1.161.155] with user [webmaster@truc.com] auth [LOGIN]: 2 NO [ALERT] LOGIN DENIED -- COUNTRY IS BLACKLISTED
+
+A. The message "[ALERT] LOGIN DENIED -- COUNTRY IS BLACKLISTED" comes directly 
+   from the IMAP server at 10.1.161.155
+
+It looks like that the 10.1.161.155 imap server filters incoming 
+connections based on their IPS. In that case, here are the current 
+IPs of the imapsync online service (September 2022):
+
+ks5.lamiral.info      has address      91.121.221.224
+ks5ipv6.lamiral.info  has IPv6 address 2001:41d0:2:84e0::1
+ks6.lamiral.info      has address      5.39.87.81
+ks6ipv6.lamiral.info  has IPv6 address 2001:41d0:8:9951::1
+ks7.lamiral.info      has address      5.135.177.225
+ks7ipv6.lamiral.info  has IPv6 address 2001:41d0:8:b8e1::1
+i050.lamiral.info     has address      213.32.72.139
+i050ipv6.lamiral.info has IPv6 address 2001:41d0:302:1000::155d
+vp3.lamiral.info      has address      51.178.81.27
+vp3ipv6.lamiral.info  has IPv6 address 2001:41d0:404:200::4d81
+vp4.lamiral.info      has address      51.38.34.201
+vp4ipv6.lamiral.info  has IPv6 address 2001:41d0:305:2100::4c46
+
+The imapsync online service may contact your imap server with one
+or more of those IP addresses.
+
+CPhulk of Cpannel has a buildin country block that can cause this behavior.
+Later on you may encountered issues with maximum sessions per ip.
+You can change that within cPannel.
+
+https://support.cpanel.net/hc/en-us/articles/4406663082519-What-is-cPHulk-
+https://docs.cpanel.net/whm/security-center/cphulk-brute-force-protection/
 
 =====================================================================
 =====================================================================

FAQ.d/FAQ.Oracle-UCS.txt

@@ -1,23 +0,0 @@
-#!/bin/cat
-$Id: FAQ.Oracle-UCS.txt,v 1.5 2018/05/24 11:34:30 gilles Exp gilles $
-
-This document is also available online at
-https://imapsync.lamiral.info/FAQ.d/
-https://imapsync.lamiral.info/FAQ.d/FAQ.Oracle-UCS.txt
-
-
-=======================================================================
-       Imapsync tips for Oracle-UCS. Specific issues.
-=======================================================================
-
-Oracle-UCS was previously Sun JES, IPlanet, etc.
-
-"NO Message contains NUL characters"
---skipmess 'm/(\x00)+\Z/'
-
-"Message contains invalid header"
---skipmess 'm/[\x80-\xff]/' 
-
-
-=======================================================================
-=======================================================================

FAQ.d/FAQ.POP3.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.POP3.txt,v 1.3 2018/05/24 11:34:30 gilles Exp gilles $
+$Id: FAQ.POP3.txt,v 1.6 2022/04/05 13:48:52 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -24,12 +24,13 @@ Q. Can I use imapsync to migrate emails from pop3 server to imap server?
 
 R1. No, but you can migrate emails from a pop3 server to an imap server 
 with the command line tool pop2imap:
-http://www.linux-france.org/prj/pop2imap/
+http://linux-france.tk/prj/pop2imap/
+http://linux-france.tk/prj/pop2imap/README
 
 R2. Yes, sometimes, because many pop3 servers runs in parallel
 with an imap server on exactly the same mailboxes. They serve 
-the same INBOX (imap serves INBOX and several other folders, 
-pop3 serves only INBOX).
+the same INBOX, ie, imap serves INBOX and several other folders, 
+pop3 serves only INBOX.
 So have a try with imapsync on the same host1.
 
 =======================================================================
@@ -38,25 +39,80 @@ Q. How can I handle those terrible POP-leaving-a-copy users?
    their emails after a migration, which is a big pain and 
    causes hundreds of support calls. 
 
+R1. Context and condition: you have access to the POP email client tool
+and the user WILL STAY using POP.
 
-R. A discussion about this issue
-http://www.linux-france.org/prj/imapsync_list/msg02622.html
-http://www.linux-france.org/prj/imapsync_list/msg02623.html
+POP email client tools can leave or delete the messages after pulling 
+them. It's very often a configurable behavior. Configure the email client
+tool to delete the messages already pulled and then they won't never 
+be pulled again from the server to the client. 
+
+If you can't access to the POP email client tool, see R2 below.
+
+Problem in the future: INBOX will be seen as empty when the email client
+go to IMAP (unless the email client tool allows then an upload
+synchronization, but don't count on that...)
+
+
+R2. Context and condition: you know which users use POP.
+
+If the pop mailboxes are known then all the INBOX messages can be moved 
+in an subfolder called INBOX_backup for example. To help you do this,
+the tool pop2imap transfers messages from POP to IMAP, without duplicates, 
+without deleting the messages transferred, by default, and can put them 
+in any subfolder you want to. It can also delete them and, that way,
+do what the email client tool don't or can't do, for any reason. 
+We're here to do a good job with what we have, do we?
+
+In case of Dovecot for example, the magic can be there since already 
+fetched messages via POP are seen as \Seen in IMAP but the new messages are 
+seen as new messages, thanks to the Dovecot developer Timo's cleverness! 
+So it is possible to sync back only the new messages to INBOX, 
+with "imapsync ... --search UNSEEN ...", and then  POP users won't see the old 
+messages come back but they will see the new messages and fetch them via POP.
+
+
+R3. Context and condition: you know nothing
+
+Do R2 without the sync back of UNSEEN messages. It can't hurt to have a backup
+of INBOX in a subfolder for IMAP or POP users.
+
+R4. Here are some points, an analysis that made me propose the not 
+too bad but not very good R2 answer.
+
+There was a discussion about this issue on the imapsync mailing-list:
+http://linux-france.tk/prj/imapsync_list/msg02622.html
+http://linux-france.tk/prj/imapsync_list/msg02623.html
 My reply was not very good
-http://www.linux-france.org/prj/imapsync_list/msg02624.html
+http://linux-france.tk/prj/imapsync_list/msg02624.html
 It's because I wrote pop2imap and it uses a stateless
-mechanism to avoid duplicates, I wasn't aware of UIDL.
-
-Here is a better response to handle and fix this problem, I hope:
-If the pop mailboxes are known and known to be regularly fetched, then
-all the INBOX messages, or at least the "old" ones, can be moved in an
-subfolder INBOX_back. It requires to have "access" to the POP
-sessions. The POP server log file is usually enough to know which
-account opened a pop connection, and when.
+mechanism to avoid duplicates, I wasn't aware of UIDL when I wrote pop2imap,
+and later when I entered the discussion.
 
 See also Rick Sanders discussion about this issue:
 https://www.emailquestions.com/threads/how-to-prevent-duplicate-emails-after-server-migration-for-pop3-users-with-leave-mail-on-server.8109/
 
+Quoting Timo Sirainen on
+https://wiki2.dovecot.org/Migration
+
+"If a client already saw changed UIDLs and decided to start
+re-downloading mails, it's unlikely there is anything you can do to
+stop it. Even going back to your old server is unlikely to help at
+that point."
+
+"Some (many?) POP3 clients also require that the message ordering is
+preserved."
+
+"Some clients re-download all mails if you change the hostname in the
+client configuration. Be aware of this when testing."
+
+Quoting 
+https://wiki.dovecot.org/POP3Server
+
+"Flag changes
+By default when a message is RETRed, \Seen flag is added to it. 
+POP3 itself doesn't support flags, but if the mailbox is opened 
+with IMAP (eg. from webmail) it's shown as seen."
 
 =======================================================================
 =======================================================================

FAQ.d/FAQ.Passwords_on_Mac.txt

@@ -0,0 +1,38 @@
+#!/bin/cat
+$Id: FAQ.Passwords_on_Mac.txt,v 1.1 2020/02/12 11:25:48 gilles Exp gilles $
+
+This document is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Mac.txt
+
+
+=======================================================================
+              Imapsync issues with passwords on Mac OS X.
+=======================================================================
+
+
+=======================================================================
+Q. On Mac, some passwords contain some weird *()",;&~ characters. 
+   Login fails.
+
+R1. Enclose the password within single-quotes in the imapsync
+    command line:
+
+    imapsync ... --password1 'passw*()",;&rd~'
+
+CAVEAT: Be careful to use quotes and double-quotes characters encoded
+from the ASCII table. How to be sure of that? 
+Mac OS X has a "use smart quotes" option. This is particularly handy 
+when typesetting and the system automatically substitutes opening and 
+closing quotes. It isn't handy when editing the imapsync command line.
+
+Disable "use smart quotes and dashes" in the Keyboard configuration,
+located in the "Text" tab. It will fix this unfortunate character 
+transformation. You can put it back later.
+
+R2. Change the password in order to keep only f...ing normal characters,
+    ie, alphanumerics like aAzZ-_09, and minus and underscore.
+    Make it long and random if strong security is your concern.
+
+=======================================================================
+=======================================================================

FAQ.d/FAQ.Passwords_on_Unix.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Passwords_on_Unix.txt,v 1.3 2019/01/10 13:27:28 gilles Exp gilles $
+$Id: FAQ.Passwords_on_Unix.txt,v 1.7 2020/12/04 11:46:54 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -12,19 +12,29 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt
 
 
 =======================================================================
-Q. On Unix, some passwords contain some *(),;& characters. Login fails.
+Q. On Unix, some passwords contain some weird *()$,";&~ characters.
+   Login fails.
 
 R1. Enclose the password within single-quotes in the imapsync
     command line:
 
-    imapsync ... --password1 'passw*(),;&rd'
+    imapsync ... --password1 'passw*()`"$,;&rd~'
 
-R2. If R1 fails, with very old imapsync or old Mail::IMAPClient Perl module,
+
+R2. Change the password to keep only f...ing normal characters.
+    Make it long and random if strong security is your concern.
+
+R3. If R1 fails, with very old imapsync or old Mail::IMAPClient Perl module,
     try also using double-quotes within single-quotes. It will enclose
     the password within double-quotes in the imap LOGIN command:
 
-    imapsync ... --password1 '"passw*(),;&rd"'
-
+    imapsync ... --password1 '"passw*()`$,;&rd~"'
 
+Old Mail::IMAPClient 3.37 is a bad horse having this issue solved
+by this double-quotes trick.
+fixed in 3.38 https://metacpan.org/changes/distribution/Mail-IMAPClient
+version 3.38: Tue Feb  9 02:48:21 UTC 2016
+        - rt.cpan.org#110273: failure to quote password values
+          (regression introduced in 3.36 via fix for rt.cpan.org#100601)
 =======================================================================
 =======================================================================

FAQ.d/FAQ.Passwords_on_Windows.txt

@@ -20,7 +20,7 @@ http://www.dostips.com/forum/viewtopic.php?f=3&t=1733
 
 =======================================================================
 Q. On Windows, some passwords contain $ or other special characters
-   like one more more of the nine characters $%&<>|^"! 
+   like one more of the nine characters $%&<>|^"! 
    Login fails. 
 
    

FAQ.d/FAQ.Principles.txt

@@ -0,0 +1,49 @@
+
+$Id: FAQ.Principles.txt,v 1.10 2021/04/18 13:00:00 gilles Exp gilles $
+
+This documentation is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Principles.txt
+
+
+=======================================================================
+             Imapsync principles and design ideas  
+=======================================================================
+
+
+Questions answered in this FAQ are:
+
+Q. Give me some design ideas of imapsync
+
+Now the questions again with their answers.
+
+=======================================================================
+Q. Give me some design ideas of imapsync
+
+R.
+
+* No configuration file
+* Stateless
+* Rsync in mind
+* Reliable
+* Robust
+* The user is not an expert, help him with:
+  * Good behavior by default. 
+  * No harm by default. 
+  * Prefer fixing an issue in the code than documenting the issue
+  * Inline help suggestions
+
+Consequences of those design ideas:
+
+* Stateless: 
+
+   * A sync can be done by another tool before or after imapsync, it is
+   safe as long as the other tools don't modify the messages' headers 
+   "Message-Id:" and "Received:", or at least "Message-Id:".
+
+   * Ready to run in a container.
+
+
+
+=======================================================================
+=======================================================================

FAQ.d/FAQ.Release_Checklist.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Release_Checklist.txt,v 1.10 2019/02/14 15:52:44 gilles Exp gilles $
+$Id: FAQ.Release_Checklist.txt,v 1.27 2022/09/16 05:35:27 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -10,21 +10,69 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Release_Checklist.txt
 =======================================================================
 
 
-Checklist before release a new release:
+Checklist before making a new release:
+
+- Make "imapsync --tests" pass on Linux
+- Make "imapsync --tests" pass on Mac
+- Make "imapsync --tests" pass on Windows 32 bits
+- Make "imapsync --tests" pass on Windows 64 bits
+
+make linuxtests 
+make win32tests
+make win64tests 
+make mactests
+
+or 
+
+make -j linuxtests win32tests win64tests mactests
+
+- Make "imapsync --testslive" pass on Linux
+- Make "imapsync --testslive" pass on Mac
+
+
+make linuxtestslive mactestslive mactestslive6
+
 
 - Add a new section in S/news.shtml reading "rlog imapsync"
+- In case of a new dependency, add it to all the INSTALL/* files.
 - Generate the README
+
+make doc
+
 - Run a spell checker on the README
 - Read the README again slowly. Fix all issues, all.
 - Read the OPTIONS section of README and read it very slowly
 - Read slowly README_Windows.txt
 - Read slowly the TUTORIAL_Unix file in html
-- Review the newsletter by running:
-    m4 -P W/ml_announce.in
 
+- Make "rcsdiff imapsync" give nothing relevant and exit 0
+
+- make ci pass.
+
+- Review the newsletter by running:
+    m4 -P W/ml_announce.in.txt
+
+- Update the file W/rsync_exclude_dist.txt
+- Update the file .gitignore
+
+- Review the TODO file and mark done what is done.
 - Review the general FAQ.d/FAQ.General.txt
+- Report values of --gmail1 --gmail2 to FAQ.Gmail.txt
+- Report values of --exchange* to FAQ.Exchange.txt
+- Report values of --office*   to FAQ.Office365.txt
+
+- /X verify direct cgi
+- /X verify under noscript with firefox
+- /X verify under private with chrome
+
+- Check the binaries imapsync.exe imapsync_32bit.exe with
+  https://www.virustotal.com/
+  https://www.metadefender.com/
+
 
 - make dist
+- verify on Windows the zip extraction and simple execution of imapsync_example.bat
+- verify binary version matches imapsync script version
 - make publish
 - verify VERSION is uploaded
 

FAQ.d/FAQ.SSL_errors.txt

@@ -1,57 +1,88 @@
 #!/bin/cat
-$Id: FAQ.SSL_errors.txt,v 1.6 2018/05/24 11:34:30 gilles Exp gilles $
+$Id: FAQ.SSL_errors.txt,v 1.18 2022/01/14 21:20:37 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
 https://imapsync.lamiral.info/FAQ.d/FAQ.SSL_errors.txt
 
 
-=======================================================================
+======================================================================
                Imapsync SSL errors
-=======================================================================
+======================================================================
 
 Questions answered in this FAQ are:
 
+Q. What is the error
+   DEBUG: .../IO/Socket/SSL.pm:1177: global error: Undefined SSL object
+
 Q. What are the errors
-  DEBUG: .../IO/Socket/SSL.pm:1165: local error: SSL write error
-  or
-  DEBUG: .../IO/Socket/SSL.pm:1088: local error: SSL read error
+   DEBUG: .../IO/Socket/SSL.pm:1165: local error: SSL write error
+   or
+   DEBUG: .../IO/Socket/SSL.pm:1088: local error: SSL read error
 
 Q. What can I do to avoid those "SSL read/write errors"?
 
-Q. SSL connect attempt failed SSL 
-   routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
-
-=======================================================================
 Q. What are the errors
-  DEBUG: .../IO/Socket/SSL.pm:1165: local error: SSL write error
-  or
-  DEBUG: .../IO/Socket/SSL.pm:1088: local error: SSL read error
+   SSL connect attempt failed SSL
+   routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
+   or 
+   SSL connect attempt failed SSL
+   SSL routines:ssl_choose_client_version:unsupported protocol
+
+Q. What is the error
+   fatal SSL error: SSL connect attempt failed with unknown error 
+   SSL wants a read first
+
+Q. How to see the certificate and identify problems in it?
+
+Now the questions again with their answers.
+
+======================================================================
+Q. What is the error
+   DEBUG: .../IO/Socket/SSL.pm:1177: global error: Undefined SSL object
+   
+R. It's a fake error from the Perl Module IO::Socket::SSL
+   Imapsync works well despite this fake warning but it's disturbing
+   when you encounter errors due to something else, you believe it's
+   the issue but no, it's something else to deal with.
+   
+   This fake error is fixed in  IO::Socket::SSL release 2.073
+   https://metacpan.org/dist/IO-Socket-SSL/changes
+   "fix #110 - prevent internal error warning in some cases"
+   https://github.com/noxxi/p5-io-socket-ssl/issues/110
+   
+   imapsync.exe release 2.178 uses this fixed 2.073 IO::Socket::SSL
+
+======================================================================
+Q. What are the errors
+   DEBUG: .../IO/Socket/SSL.pm:1165: local error: SSL write error
+   or
+   DEBUG: .../IO/Socket/SSL.pm:1088: local error: SSL read error
 
 
-R1.Like they claim, those errors are SSL errors. SSL is not directly
-   done by imapsync but by an underlying Perl module called
-   IO::Socket::SSL. Those errors arise sometimes and sometimes
-   they form a serie that ends with imapsync auto-abortion.
-   Those errors happen with some hosts but not with others,
-   it's often Exchange or Office365. I don't know what exactly happens.
-   Those errors happen more often on Windows than on Linux.
+R1. As they claim, those errors are SSL errors. SSL is not directly
+done by imapsync but by an underlying Perl module called
+IO::Socket::SSL. Those errors arise sometimes and sometimes
+they form a series that ends with imapsync auto-abortion.
+Those errors happen with some hosts but not with others,
+it's often Exchange or Office365. I don't know what exactly happens.
+Those errors happen more often on Windows than on Linux.
 
 
-=======================================================================
+======================================================================
 Q. What can I do to avoid those "SSL read/write errors"?
 
 R0. Windows users: upgrade to imapsync.exe release 1.836 (or next ones)
-    Those errors don't appear with recent releases, post 1.836
+Those errors appear less often with imapsync releases post 1.836
 
 R1. Remove all ssl/tls encryption
 
   imapsync ...   --nossl1 --notls1 --nossl2 --notls2
 
 R2. If you don't want to quit encryption, rerun imapsync until the 
-    complete sync is over. Those errors are not at the same place 
-    each time, so imapsync will sync remaining messages at each run
-    until none remains.
+complete sync is over. Those errors are not at the same place 
+each time, so imapsync will sync the remaining messages at each run
+until none remains.
 
 R3. Run imapsync on a Linux machine, a VM is ok, there are less
     SSL errors on Unix.
@@ -63,22 +94,163 @@ R5. Set up a ssltunnel proxy to the host.
     Read the file FAQ.Security.txt for an example to set up
     a ssltunnel proxy.
 
-=======================================================================
-Q. SSL connect attempt failed SSL 
+======================================================================
+Q. What are the errors
+   SSL connect attempt failed SSL
    routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
+   or 
+   SSL connect attempt failed SSL
+   SSL routines:ssl_choose_client_version:unsupported protocol
 
-R. Use: 
 
-   imapsync ... --sslargs1 'SSL_cipher_list=DEFAULT' 
+R1. Use: 
+
+   imapsync ...  --sslargs1 SSL_cipher_list=DEFAULT
    or    
-   imapsync ... --sslargs2 'SSL_cipher_list=DEFAULT' 
+   imapsync ...  --sslargs2 SSL_cipher_list=DEFAULT
 
    depending on where the error occurs, host1 or host2 or both.
-   
-=======================================================================
-=======================================================================
+
+R2. If it doesn't work, I let you try other things, 
+    I quote the "SSL_version" section of
+    https://metacpan.org/pod/IO::Socket::SSL (Module version: 2.066)
+
+    imapsync ...  --sslargs1 SSL_cipher_list=DEFAULT
+    imapsync ...  --sslargs1 SSL_version=SSLv2
+    imapsync ...  --sslargs1 SSL_version=SSLv23
+    imapsync ...  --sslargs1 SSL_version=SSLv3
+    imapsync ...  --sslargs1 SSL_version=TLSv1
+    imapsync ...  --sslargs1 SSL_version=TLSv1_1
+    imapsync ...  --sslargs1 SSL_version=TLSv1_2
+    imapsync ...  --sslargs1 SSL_version=TLSv1_3
+
+Those examples are for host1. For host2, use --sslargs2 instead.
+Feedback on what worked for you is welcome!
+
+A loop to check every version and print the good ones:
+
+for v in SSLv2 SSLv23 SSLv3 TLSv1 TLSv1_1 TLSv1_2 TLSv1_3; do
+        imapsync ...  --sslargs1 SSL_version=$v && GOOD="$GOOD $v"
+done
+echo "$GOOD"
+
+I reproduce below the documentation of the underlying Perl 
+module IO::Socket::SSL used by imapsync:
+
+https://metacpan.org/pod/IO::Socket::SSL
+...
+SSL_version
+
+Sets the version of the SSL protocol used to transmit data.
+'SSLv23' uses a handshake compatible with SSL2.0, SSL3.0 and TLS1.x, 
+while 'SSLv2', 'SSLv3', 'TLSv1', 'TLSv1_1', 'TLSv1_2', or 'TLSv1_3' 
+restrict handshake and protocol to the specified version. 
+All values are case-insensitive. Instead of 'TLSv1_1', 'TLSv1_2', and 'TLSv1_3' 
+one can also use 'TLSv11', 'TLSv12', and 'TLSv13'.
+
+Support for 'TLSv1_1', 'TLSv1_2', and 'TLSv1_3' 
+requires recent versions of Net::SSLeay and openssl.
+
+Independent from the handshake format you can limit to set of 
+accepted SSL versions by adding !version separated by ':'.
+The default SSL_version is 'SSLv23:!SSLv3:!SSLv2' which means, 
+that the handshake format is compatible to SSL2.0 and higher, 
+but that the successful handshake is limited to TLS1.0 and higher, 
+that is no SSL2.0 or SSL3.0 because both of these versions have
+serious security issues and should not be used anymore.
+
+You can also use !TLSv1_1 and !TLSv1_2 to 
+disable TLS versions 1.1 and 1.2 while still allowing TLS version 1.0.
+
+Setting the version instead to 'TLSv1' might break interaction 
+with older clients, which need and SSL2.0 compatible handshake.
+
+On the other side some clients just close the connection 
+when they receive a TLS version 1.1 request. 
+In this case setting the version 
+to 'SSLv23:!SSLv2:!SSLv3:!TLSv1_1:!TLSv1_2' might help.
+
+======================================================================
+Q. What is the error
+   fatal SSL error: SSL connect attempt failed with unknown error 
+   SSL wants a read first
+
+R. If you're using --ssl1 or --ssl2, try instead --tls1 or --tls2
+
+======================================================================
+Q. How to see the certificate and identify problems in it?
+
+R. Use the command openssl like this:
+
+  echo | openssl s_client -crlf  -connect imap.gmail.com:993
+
+  echo | openssl s_client -crlf  -connect test1.lamiral.info:993
+
+and examine carefully the content, the  "verify return:" lines,
+the chain. Sometimes, the server certificate is ok but not the whole 
+chain of certificates so the certification fails.
+
+Here is an example.
+
+One of the certificate is expired:
+
+echo | openssl s_client -crlf -connect test1.lamiral.info:993
+CONNECTED(00000003)
+depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
+verify return:1
+depth=1 C = US, O = Let's Encrypt, CN = R3
+verify return:1
+depth=0 CN = test1.lamiral.info
+verify error:num=10:certificate has expired
+notAfter=Apr 11 10:14:05 2021 GMT
+verify return:1
+depth=0 CN = test1.lamiral.info
+notAfter=Apr 11 10:14:05 2021 GMT
+verify return:1
+---
+Certificate chain
+ 0 s:/CN=test1.lamiral.info
+   i:/C=US/O=Let's Encrypt/CN=R3
+ 1 s:/C=US/O=Let's Encrypt/CN=R3
+   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
+---
+Server certificate
+-----BEGIN CERTIFICATE-----
+MIIFKjCCBBKgAwIBAgISBHYZCE3qSTIlvq97HI5TpBeAMA0GCSqGSIb3DQEBCwUA
+MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
+EwJSMzAeFw0yMTAxMTExMDE0MDVaFw0yMTA0MTExMDE0MDVaMB0xGzAZBgNVBAMT
+EnRlc3QxLmxhbWlyYWwuaW5mbzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
+ggEBAMUTJVdrTl86nDI2yO6Vz5l1qxMMPqJylQcgi9vDHpwsnUq5HGPv+qZNhM69
+...
 
 
+After an complete server update ("apt update && apt upgrade && /etc/init.d/dovecot restart"):
 
+echo | openssl s_client -crlf -connect test1.lamiral.info:993
+CONNECTED(00000003)
+depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
+verify return:1
+depth=1 C = US, O = Let's Encrypt, CN = R3
+verify return:1
+depth=0 CN = test1.lamiral.info
+verify return:1
+---
+Certificate chain
+ 0 s:/CN=test1.lamiral.info
+   i:/C=US/O=Let's Encrypt/CN=R3
+ 1 s:/C=US/O=Let's Encrypt/CN=R3
+   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
+---
+Server certificate
+-----BEGIN CERTIFICATE-----
+MIIFKTCCBBGgAwIBAgISBD4QN3cfB1JpTm75oVrkkAElMA0GCSqGSIb3DQEBCwUA
+MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
+EwJSMzAeFw0yMTAzMTIxODQxMTJaFw0yMTA2MTAxODQxMTJaMB0xGzAZBgNVBAMT
+EnRlc3QxLmxhbWlyYWwuaW5mbzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
+ggEBANuPNbYLPMZ4vPa9NBoHAUdIXqpi0eqdXMXd2sT+qRmqxS5ihr999BHOROcr
+...
 
+Champagne!
 
+======================================================================
+======================================================================

FAQ.d/FAQ.Security.txt

@@ -1,45 +1,122 @@
 #!/bin/cat
-# $Id: FAQ.Security.txt,v 1.19 2018/09/13 17:53:45 gilles Exp gilles $
+# $Id: FAQ.Security.txt,v 1.26 2021/07/11 12:11:38 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
 https://imapsync.lamiral.info/FAQ.d/FAQ.Security.txt
 
-
 =======================================================================
       Imapsync tips about security. Issues and solutions.
 =======================================================================
 
-SSL: going to encryption before the imap session start. 
-     It is on port 993.
+Questions answered in this FAQ are:
 
-TLS: going to encryption after the imap session start but before
-     the credential are sent. It is on port 143.
+Q. Is running this program a secure method of transferring emails?  
+   Are there any security concerns?
+
+Q. Does imapsync refer to SSL security protocols with --ssl1 and --ssl2 
+   and does it refer to TLS security protocols with the --tls1 and --tls2
+   options? 
+   Short answer: No.
+
+Q. I noticed that the online UI has no option for TLS/SSL. 
+   Is this secure? 
+   Is this more secure than using the .bat file on my computer?
+
+Q. Are transferred emails/attachments stored on any other 
+   server/location aside from my originating/destination server(s)?
+
+Q. Other than changing passwords on the originating/destination email
+   accounts once the relevant emails have been moved,
+   are there any other security tips I should know?
+
+Q. I need to transfer mail from an imap server to another imap server.
+   Which ports need to be open on the firewall to make this possible?
+
+Q. Does imapsync support the IMAP command STARTTLS?
+
+Q. Does imapsync support IMAP over SSL/TLS (IMAPS)?
+
+Q. How can I test an ssl/tls imap connection without imapsync?
+
+Q. How can I manually test login using --ssl?
+
+Q. Imapsync used to use SSL_VERIFY_PEER now it uses SSL_VERIFY_NONE.
+   How can I change this back to the more secure SSL_VERIFY_PEER?
+
+Q: How can I have an imaps server?
+
+Now the questions again with their answers.
 
 =======================================================================
 Q. Is running this program a secure method of transferring emails?  
    Are there any security concerns?
 
-R. Well, it depends. Use encryption, secure the access to the host running
-   imapsync and everything shall be safe.
+R. Well, it depends. Use encryption and secure access to the host running
+   imapsync then everything shall be safe.
 
 =======================================================================
-Q. I noticed that the online UI has no option for TLS/SSL 
-   Is this secure? Is this more secure that using the .bat file 
-   on my computer?
+Q. Does imapsync refer to SSL security protocols with --ssl1 and --ssl2 
+   and does it refer to TLS security protocols with the --tls1 and --tls2
+   options? 
+   Short answer: No.
 
-R1. The online UI does TLS/SSL imap connections if the imap
-    servers support TLS/SSL. If you are concerned by security then 
-    using the .bat file or .sh on your computer should be more secure 
-    since you can examine and secure it by yourself, no matter high is
-    your paranoïd spirit. The online UI security is mine, I am
-    concerned by security, not to the upmost high level possible 
-    but I won't give you direct access to the host to discover my level.
-    With a good guy spirit, feel free to try to break the online UI 
-    security and report me any security issue you encounter, I'll do my
-    best to fix them as soon as possible. Drop me a note before
-    starting because I may detect a sort of abuse and ban 
-    definitively your IPs.
+R. No.
+
+Imapsync behaviour:
+
+--ssl: Goes to encryption before the imap session start. 
+     The connection is on port 993. 
+     Can use the TLS or SSL security protocols.
+     Fails if encryption cannot be established.
+
+--tls: Goes to encryption after the imap session start but before the
+     credential are sent. 
+     The connection is on port 143.  
+     Can use the TLS or SSL securityprotocols.  
+     Fails if encryption cannot be established.
+
+Explanation:
+
+To force transferring emails over an encrypted connection, you can use
+the imapsync parameters --ssl1 and --ssl2 or the parameters --tls1 and
+--tls2, they all force the connection to be encrypted by the security
+protocols series SSL/TLS.
+
+In a security context, SSL refers to all deprecated Secure Sockets
+Layer protocols. TLS refers to the SSL successors, Transport Layer
+Security protocols. But TLS 1.0 and TLS 1.1 are also deprecated.
+
+Current endorsed versions of TLS are only TLS 1.2 and TLS 1.3 (July 2021).
+
+The whole story is detailed here:
+https://en.wikipedia.org/wiki/Transport_Layer_Security
+
+Options --ssl1 and --ssl2 are a little more paranoid than --tls1 and
+--tls2 because they verify that the hostname of the certificate is the
+same as the one used by imapsync. Other than that, the security
+behavior is the same.
+
+=======================================================================
+Q. I noticed that the online UI has no option for TLS/SSL. 
+   Is this secure? 
+   Is this more secure than using the .bat file on my computer?
+
+R1. The online UI does TLS/SSL imap connections if the imap servers
+support TLS/SSL. 
+
+If you are concerned about security then using the .bat file or .sh on
+your computer should be more secure since you can examine and secure
+it by yourself, no matter high is your paranoid spirit compared to
+mine.
+
+The online UI security is mine, I am concerned by security, not to the
+utmost high level possible but I won't give you direct access to the
+host to discover my level.  With a good guy spirit, feel free to try
+to break the online UI security and report me any security issue you
+encounter, I'll do my best to fix them as soon as possible. Drop me a
+note before starting because I may detect a sort of abuse and ban your
+IPs.
 
 =======================================================================
 Q. Are transferred emails/attachments stored on any other 
@@ -48,15 +125,15 @@ Q. Are transferred emails/attachments stored on any other
 R. No!
 
 =======================================================================
-Q. Other than changing passwords on the originating/destination email 
-   accounts once the relevant emails have been moved, 
+Q. Other than changing passwords on the originating/destination email
+   accounts once the relevant emails have been moved,
    are there any other security tips I should know?
 
-R. Secure the host where imapsync is running since credentials are
-   on it.
+R. Yes. Secure the host where imapsync is running since credentials
+   are on it.
 
 =======================================================================
-Q. I need to transfer mail from an imap server to an other imap server. 
+Q. I need to transfer mail from an imap server to another imap server.
    Which ports need to be open on the firewall to make this possible?
 
 R. It depends. Open either:
@@ -64,29 +141,31 @@ R. It depends. Open either:
     * port 993 in ssl mode (--ssl1 or --ssl2)
 
 =======================================================================
-Q. Does imapsync support IMAP TLS?
+Q. Does imapsync support the IMAP command STARTTLS?
 
 R1. Yes. 
-   Use --tls1 and/or --tls2 options:
 
-  --tls1 tells imapsync to use tls on host1.
-  --tls2 tells imapsync to use tls on host2.
+   Use --tls1 and --tls2 options:
 
-R2. Since imapsync release 1.755 TLS mode is activated automatically
-    if the server announce it supports it, STARTTLS inside the response 
-    to CAPABILITY, and if neither --notls nor --ssl is explicitely
-    mentioned on the command line options.
+  --tls1 tells imapsync to use STARTTLS on host1.
+  --tls2 tells imapsync to use STARTTLS on host2.
+
+R2. Since imapsync release 1.755 STARTTLS mode is activated
+    automatically when the server announces that it supports it by
+    listing STARTTLS inside the response to the CAPABILITY command.
+    If either --notls or --ssl are explicitly mentioned on the
+    command-line options then STARTTLS won't be done.
 
 =======================================================================
-Q. Does imapsync support IMAP over SSL (IMAPS)?
+Q. Does imapsync support IMAP over SSL/TLS (IMAPS)?
 
 R. Yes natively since release 1.161. 
    Still, there are 2 ways, at least, to use ssl:
 
 a) Use native --ssl1 and/or --ssl2 options
 
-  --ssl1 tells imapsync to use ssl on host1.
-  --ssl2 tells imapsync to use ssl on host2.
+  --ssl1 tells imapsync to use ssl/tls on host1.
+  --ssl2 tells imapsync to use ssl/tls on host2.
 
 
 b) Use stunnel
@@ -108,18 +187,18 @@ b) Use stunnel
    If the local port 143 is already taken then use a free one, 
    like 10143 for example.
 
-c) Other example for accessing gmail with no local root access 
-   to open port 143
+c) Another example for accessing Gmail with no local root access 
+   to open port 143:
 
    stunnel3 -P '' -c -d 9993 -r imap.gmail.com:993 -f 
 
-Then, to access gmail as host2 use:
+Then, to access Gmail as host2 use:
 
   imapsync ... --host2 localhost --port2 9993 --nossl2 
 
 
 =======================================================================
-Q.How can I test an ssl imap connection without imapsync?
+Q.How can I test an ssl/tls imap connection without imapsync?
 
 R1. Use either ncat or telnet-ssl or openssl commands like in the 
     following examples with imap.gmail.com server:
@@ -133,37 +212,39 @@ R1. Use either ncat or telnet-ssl or openssl commands like in the
 The previous commands are interactive, hit ctrl-c
 to finish them. If you want to finish automatically, then use: 
 
-  { sleep 2; echo "a logout"; sleep 1; } | ncat --ssl -C  imap.gmail.com 993
+  { sleep 2; echo "A LOGOUT"; sleep 1; } | ncat --ssl -C  imap.gmail.com 993
 
 
 =======================================================================
-Q. How can I manually test a login via ssl?
+Q. How can I manually test login using --ssl?
 
 R. Use either ncat or telnet-ssl or openssl commands like in the 
     following examples with imap.gmail.com server:
 
   ncat --ssl -C imap.gmail.com 993
+
   telnet-ssl -z ssl  imap.gmail.com 993
+
   openssl s_client -crlf  -connect imap.gmail.com:993
 
-Tipical dialog for an imap LOGIN command:
+Typical dialog for an imap LOGIN command:
   
 * OK Gimap ready for requests from 78.196.254.58 q1mb175739668wix
-a LOGIN "gilles.lamiral@gmail.com" "secret" 
+A1 LOGIN "gilles.lamiral@gmail.com" "secret" 
 * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE ... ESEARCH
-a OK gilles.lamiral@gmail.com Gilles Lamiral authenticated (Success)
-b LOGOUT
+A1 OK gilles.lamiral@gmail.com Gilles Lamiral authenticated (Success)
+A2 LOGOUT
 * BYE LOGOUT Requested
-b OK 73 good day (Success)
+A2 OK 73 good day (Success)
 
 The client part you have to type is 
-a LOGIN ... 
-b LOGOUT
-while replacing ... by your credentials values.
-
+A1 LOGIN ... 
+A2 LOGOUT
+while replacing ... by your credentials values,
+other lines are the server responses.
 
 =======================================================================
-Q.How can I test an tls imap connection without imapsync?
+Q. How can I test a STARTTLS imap connection without imapsync?
 
 R1. Use openssl command like the following example with 
     an outlook.office365.com server:
@@ -177,13 +258,12 @@ to finish them. If you want to finish automatically, then use:
 
 Replace outlook.office365.com with your imap server name.
 
-
 ======================================================================
 Q. Imapsync used to use SSL_VERIFY_PEER now it uses SSL_VERIFY_NONE.
    How can I change this back to the more secure SSL_VERIFY_PEER?
 
 
-R2. After imapsync 1.673, 
+R. After imapsync 1.673, 
     to set SSL_verify_mode to SSL_VERIFY_PEER on host1
     and    SSL_verify_mode to SSL_VERIFY_NONE on host2
 
@@ -191,19 +271,35 @@ R2. After imapsync 1.673,
                 --sslargs1 SSL_verify_mode=1 \
                 --sslargs2 SSL_verify_mode=0 
 
-See perldoc IO::Socket::SSL for all possibilities, also at
+See "perldoc IO::Socket::SSL" for all possibilities, also at
 http://search.cpan.org/perldoc?IO%3A%3ASocket%3A%3ASSL
 
-It might be possible you need an extra:
-                --sslargs1 SSL_ca_file=/etc/ssl/certs/ca-certificates.crt 
+It might be possible you need an extra option
+
+  --sslargs1 SSL_ca_file=/etc/ssl/certs/ca-certificates.crt 
+
 to help the ssl software verifying the server certificate.
 The file ca-certificates.crt may be elsewhere on your system, even 
 named differently.
 
-=======================================================================
-Q: How to have an imaps server?
+The imap server certificates are not checked for authenticity 
+by imapsync by default because too many imap servers are crappy 
+configured regarding certified certificates.
+
+This default behavior is chosen like this because users
+want their emails transferred, instead of being not transferred 
+because of an incompetent imap server sysadmin.
+
+I admit that this part, checking imap ssl/tls certificates, 
+could be improved from my side by including well known 
+certificates directly in imapsync.
+Drop me a note to encourage me, I'm lazy.
+
+=======================================================================
+Q: How can I have an imaps server?
+
+R. Three solutions.
 
-R. 
 a) Install one 
 
 b) or use stunnel :

FAQ.d/FAQ.SmarterMail.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.SmarterMail.txt,v 1.12 2018/05/24 11:34:30 gilles Exp gilles $
+$Id: FAQ.SmarterMail.txt,v 1.15 2021/04/30 13:02:17 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -10,6 +10,39 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.SmarterMail.txt
     Imapsync tips for SmarterMail. Specific issues and solutions.
 =======================================================================
 
+https://imapsync.lamiral.info/README
+...
+     --prefix1      str  : Remove prefix str to all destination folders,
+                           usually "INBOX." or "INBOX/" or an empty string "".
+                           imapsync guesses the prefix if host1 imap server
+                           does not have NAMESPACE capability. So this option
+                           should not be used most of the time.
+     --prefix2      str  : Add prefix to all host2 folders. See --prefix1
+
+     --sep1         str  : Host1 separator. This option should not be used
+                           most of the time.
+                           Imapsync gets the separator from the server itself,
+                           by using NAMESPACE, or it tries to guess it
+                           from the folders listing (it counts
+                           characters / . \\ \ in folder names and choose the
+                           more frequent, or finally / if nothing is found.
+     --sep2         str  : Host2 separator. See --sep1
+
+
+
+ --noabletosearch    : Makes --minage and --maxage options use the internal 
+                       dates given by a FETCH imap command instead of the 
+                       "Date:" header. Internal date is the arrival date 
+                       in the mailbox.
+                       --noabletosearch equals --noabletosearch1 --noabletosearch2
+
+ --noabletosearch1   : Like --noabletosearch but for host1.
+ --noabletosearch2   : Like --noabletosearch but for host2.
+
+ --useheader    str  : Use this header to compare messages on both sides.
+                       Example: Message-ID or Subject or Date.
+ --useheader    str    and this one, etc.
+
 
 =======================================================================
 Q. Synchronizing from SmarterMail to XXX
@@ -17,21 +50,13 @@ Q. Synchronizing from SmarterMail to XXX
 On Unix:
 imapsync --host1 imap.d1.org --user1 joe --password1 secret1 \
          --host2 imap.d2.org --user2 joe --password2 secret2 \
-         --sep1 "/" --prefix1 "" --useheader Message-Id --noabletosearch1 \
-	 --regextrans2 "s,Deleted Items,Trash," \
-	 --regextrans2 "s,Junk E-Mail,Junk," \
-         --regextrans2 "s,Sent Items,Sent," 
+         --prefix1 "" --sep1 "/" --useheader Message-Id --noabletosearch
 
 On Windows:
 imapsync.exe --host1 imap.d1.org --user1 joe --password1 secret1 ^
          --host2 imap.d2.org --user2 joe --password2 secret2  ^
-         --sep1 "/" --prefix1 "" --useheader Message-Id --noabletosearch1 ^
-         --regextrans2 "s,Deleted Items,Trash,"  ^
-	 --regextrans2 "s,Junk E-Mail,Junk,"    ^
-	 --regextrans2 "s,Sent Items,Sent,"
+         --prefix1 "" --sep1 "/" --useheader Message-Id --noabletosearch
 
-Maybe add other --regextrans2 to change folder names, for this
-see also http://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Mapping.txt
 
 =======================================================================
 Q. Synchronizing from XXX to SmarterMail
@@ -39,12 +64,12 @@ Q. Synchronizing from XXX to SmarterMail
 On Unix:
 imapsync --host1 imap.d1.org --user1 joe --password1 secret1  \
          --host2 imap.d2.org --user2 joe --password2 secret2  \
-         --sep2 "/" --prefix2 "" --useheader Message-Id --noabletosearch2
+         --prefix2 "" --sep2 "/" --useheader Message-Id --noabletosearch
 
 On Windows:
 imapsync.exe --host1 imap.d1.org --user1 joe --password1 secret1 ^
              --host2 imap.d2.org --user2 joe --password2 secret2 ^
-             --sep2 "/" --prefix2 "" --useheader Message-Id --noabletosearch2
+             --prefix2 "" --sep2 "/" --useheader Message-Id --noabletosearch
 
 
 =======================================================================

FAQ.d/FAQ.Two_Ways_Sync.txt

@@ -0,0 +1,104 @@
+$Id: FAQ.Two_Ways_Sync.txt,v 1.8 2022/07/25 14:03:49 gilles Exp gilles $
+
+This documentation is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Two_Ways_Sync.txt
+
+
+=======================================================================
+================  Two ways sync with Imapsync =========================
+=======================================================================
+
+=======================================================================
+Q. Can Imapsync do a good "two ways" sync? 
+   Short answer: no, not a good one. Why?
+
+R. Imapsync can't do good two ways syncs.
+
+A good "two ways" sync is impossible with imapsync because imapsync is
+stateless.
+
+Each time imapsync runs, it considers messages and folders as if it
+were the first time it encounters them. Imapsync looks at messages,
+flags, and folders as they are now, not considering what they were
+before the current. Imapsync has no memory outside the current running
+sync.
+
+So now, why a stateless behavior cannot handle well a two ways sync
+between an account A and an account B?
+
+The problem arises with deletions, messages deletions, folders
+deletions, or with moves, messages moves across folders, folders
+moves, and also folders renamings. Deletions and moves are ambiguous
+changes when combined with creations on the opposite side.
+ 
+For example, if a message is deleted from A by a user, then imapsync
+cannot know whether it is a message deleted from A that has to be
+deleted in B (what the user actually did) or a missing message on A
+that has to be copied from B.
+
+But if you know the answer yourself, that missing messages on one side
+A are deleted messages that have to be deleted on the other side then
+run a sync with the --delete2 option from A to B. This assumes there 
+is no new messages on B because --delete2 will delete them.
+
+If you know that the missing messages on A are messages from B
+that has to be copied to A, then run a sync from B to A.
+
+If you know it's a mixed scenario, that there are some deletions/moves
+on A, and there are some deletions/moves on B, but not the same, then
+you are in trouble and it ends up with a not very good "two ways"
+sync.  In that case, two runs of imapsync, one from A to B, one from B
+to A, put the two accounts in a synchronized state. The deletions of
+messages on either side get cancelled, deleted messages come back. A
+folder renaming on either side brings back the old folder.  It's not a
+good two ways synchronization.
+
+
+My suggestion is:
+
+a) Use a single active imap account that fits all your needs,
+   available from anywhere, at any time.  Access it from both your
+   laptop and your iphone.
+
+b) Feed this active account with foreign and passive accounts with any
+   tool that can handle the transfer, imapsync being one of them.
+
+c) Have a slave imap backup account of the active one, available also
+   from anywhere, at any time. Use it only as a failsafe account, when
+   the active one is broken or to bring back misguidely deleted
+   messages to the active account. Use any tool that can handle the
+   synchronization, imapsync being a good one of them.
+
+
+=======================================================================
+Q. What can I do instead of having two ways synchronizations?
+
+R. My suggestion is:
+
+a) Use only one single active imap account that fits all your needs,
+   available from anywhere, at any time.  Access it from all your 
+   devices, laptops, phones, tablets.
+
+b) Feed this active account with foreign and passive accounts with any
+   tool that can handle the transfer, imapsync being one of them.
+
+c) Have a slave imap backup account of the active one, available also
+   from anywhere, at any time. Use it only as a failsafe account, when
+   the active one is broken or to bring back misguidely deleted
+   messages to the active account. Use any tool that can handle the
+   synchronization, imapsync being a good one of them.
+
+
+=======================================================================
+Q. What tools can I use to handle two ways synchronizations?
+
+R. There are offlineimap and mbsync/isync tools:
+
+offlineimap  is located at http://www.offlineimap.org/
+mbsync/isync is located at https://isync.sourceforge.io/
+
+
+=======================================================================
+=======================================================================
+

FAQ.d/FAQ.UCS.txt

@@ -0,0 +1,37 @@
+#!/bin/cat
+$Id: FAQ.UCS.txt,v 1.8 2021/05/29 08:06:51 gilles Exp gilles $
+
+This document is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.UCS.txt
+
+
+=======================================================================
+       Imapsync tips for UCS. Specific issues.
+=======================================================================
+
+UCS: Univention Corporate Server
+https://www.univention.com/
+
+UCS was previously Oracle-UCS, Sun JES, IPlanet, etc. 
+The underlying code was the free open source imap software server Cyrus. 
+Now UCS uses Dovecot since UCS 4.0-2 (July 2015)
+
+Quoting the page
+https://docs.software-univention.de/cyrus-4.2.html
+"Since Univention Corporate Server version 4.0-2 Dovecot is used as 
+the default IMAP and POP3 server."
+
+So, when you deal with UCS issues, follow 
+https://imapsync.lamiral.info/FAQ.d/FAQ.Cyrus.txt
+https://imapsync.lamiral.info/FAQ.d/FAQ.Dovecot.txt
+Also:
+https://imapsync.lamiral.info/FAQ.d/FAQ.APPEND_errors.txt
+
+
+
+
+
+
+=======================================================================
+=======================================================================

FAQ.d/FAQ.Use_cache.txt

@@ -1,5 +1,5 @@
 
-$Id: FAQ.Use_cache.txt,v 1.8 2018/10/22 17:30:45 gilles Exp gilles $
+$Id: FAQ.Use_cache.txt,v 1.11 2021/04/22 13:20:18 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -7,11 +7,18 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Use_cache.txt
 
 
 =======================================================================
-               Imapsync  --usecache option
+               Imapsync  --usecache or --useuid options
 =======================================================================
 
 Questions answered in this FAQ are:
 
+Q. What is --usecache good for?
+
+Q. What is the technical principle of --usecache?
+
+Q. On Windows, with --useuid and after a reboot or a new login, a new
+   run generates duplicates. What can I do to avoid and fix that?
+
 Q. On Windows, with --useuid or --usecache a problem occurs with long
    nested folder names. The error message is:
    "No such file or directory; The filename or extension is too long"
@@ -19,7 +26,64 @@ Q. On Windows, with --useuid or --usecache a problem occurs with long
 Q. Inode problem with --usecache or --useuid on Linux
 
 
-Questions and their answers:
+Now the questions again and their answers.
+
+=======================================================================
+Q. What is --usecache good for?
+
+R. The option --usecache is useful to speed up next syncs with --usecache.
+   The first sync with --usecache between two imap accounts is as low 
+   as the first sync without --usecache, the next syncs are faster.
+
+=======================================================================
+Q. What is the technical principle of --usecache?
+
+R. Option --usecache creates a empty file per email message
+   in order to keep the UIDs mapping between account1 and account2.
+   
+   The cache is maintened at each run. Messages UIDs enregistred 
+   in the cache are not fetched again to be identified. It really
+   speeds up the process of syncing.
+   
+   The cache directory is 
+   
+       <tmpdir>/imapsync_cache/
+       
+   where <tmpdir> is the temporary directory given by the --tmpdir option
+   or the one by default, /tmp on Unix, variable on Windows.
+   
+=======================================================================
+Q. On Windows, with --useuid and after a reboot or a new login, a new
+   run generates duplicates. What can I do to avoid and fix that?
+
+R. The cache is build in a temporary directory. Imapsync uses a system
+call to get this temporary directory, it is not hardcoded and it
+is system dependent, the call garanties a write access. Sometimes on 
+Windows this directory name changes upon reboot or for any other reason. 
+Anyway, the temporary directory used by imapsync can be set by the 
+option --tmpdir and that's the way to avoid this issue. 
+The new problem is that the temporary directory must be writable, which
+is not a big deal to found.
+
+Use for example either a) b) or c)
+
+a) The current directory:
+
+  imapsync ... --tmpdir .
+
+b) The root directory:
+
+  imapsync ... --tmpdir /
+
+c) Any directory you want, like:
+
+  imapsync ... --tmpdir "C:\TEMP"
+
+Now, to fix the duplicates on host2, have a run with the option
+--delete2duplicates
+
+  imapsync ... --delete2duplicates
+
 
 =======================================================================
 Q. On Windows, with --useuid or --usecache a problem occurs with long

FAQ.d/FAQ.User_Concurrent_Access.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.User_Concurrent_Access.txt,v 1.5 2018/05/24 11:34:30 gilles Exp gilles $
+$Id: FAQ.User_Concurrent_Access.txt,v 1.6 2022/05/05 13:18:29 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -35,6 +35,23 @@ Details:
                            so --delete2 is needed to sync the deletion.
   * message flag change => will need a resync
 
+Let's more precise about timing.
+Imapsync works per folder, one folder at a time.
+
+If imapsync has already synced a folder during a run, 
+what happens to this folder now won't be synced by the current run.
+
+If imapsync has not already synced a folder during a run, 
+what happens to this folder now will be synced by the current run.
+
+If imapsync is syncing a folder during a run,
+what happens to this folder now depends on when things happen.
+Imapsync starts by listing messages and getting meta data about them,
+Message-Id, Date, Size.
+A message deleted just after the listing, won't be synced.
+A new message arriving just after the listing, won't be synced.
+
+
 * Writing at destination/host2:
   * adding folder       => no problem, unless --delete2folders is used
   * deleting folder     => will reappear by a new sync
@@ -42,5 +59,7 @@ Details:
   * deleting message    => might reappear by a new sync.
   * message flag change => will be reseted by a new sync.
 
+
+
 =======================================================================
 =======================================================================

FAQ.d/FAQ.Various_Software_Servers.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Various_Software_Servers.txt,v 1.10 2018/10/24 11:03:57 gilles Exp gilles $
+$Id: FAQ.Various_Software_Servers.txt,v 1.19 2021/01/26 14:05:23 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -13,12 +13,17 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Various_Server_Softwares.txt
 
 
 =======================================================================
-Q. From Zimbra to XXX
+Q. From or to Zimbra
 
-imapsync ... \
-	--exclude "Conversation Action Settings" \
-	--exclude "Quick Step Settings" \
-	--exclude "News Feed"
+R. Read the FAQ.Zimbra.txt 
+   https://imapsync.lamiral.info/FAQ.d/FAQ.Zimbra.txt
+
+=======================================================================
+Q. From Emailchemy (https://weirdkid.com/emailchemy/)
+
+R. Use option  --noabletosearch :
+
+  imapsync ... --noabletosearch
 
 =======================================================================
 Q. From or to HMailServer version 4.4.1.
@@ -36,16 +41,6 @@ No specific option for HMailServer 5.3.3 since NAMESPACE is supported.
 Maybe --subscribe_all will help you to see all migrated folders.
 
 
-=======================================================================
-Q. Synchronizing from Kerio Connect to XXX
-
-R. No special options required. 
-See also:
-http://www.linux-france.org/prj/imapsync_list/msg01756.html
-http://www.safetynet-it.com/it-support/mac-kerio-server-to-microsoft-exchange-2010-migration-1/
-http://www.safetynet-it.com/it-support/mac-kerio-server-to-microsoft-exchange-2010-migration-2/
-
-
 =======================================================================
 Q. from Microsoft's Exchange 2007 to Google Apps for your Domain 
    (GAFYD)
@@ -63,76 +58,10 @@ Parallells Plesk Panel for Windows requires --sep2 / --prefix2 ""
 Parallells Plesk Panel for Linux works with default parameters.
 
 =======================================================================
-Q. I'm migrating from WU to Cyrus, and the mail folders are under
-   /home/user/mail but the tool copies everything in /home/user, how
-   can i avoid that?
-
-Two solutions:
-
-R. Use
-   imapsync ... --include '^mail'
-
-R. or (better)
-   imapsync ... --subscribed --subscribe
-
-
-=======================================================================
-Q. I'm migrating from WU to Cyrus, and the mail folders are under
-   /home/user/mail directory.  When imapsync creates the folders in
-   the new cyrus imap server, it makes a folder "mail" and below that
-   folder puts all the mail folders the user have in /home/user/mail,
-   i would like to have all those folders directly under INBOX.
-
-R. Use 
-   imapsync ... --regextrans2 's/^mail/INBOX/' --dry
-   look at the simulation and if all transformations seem
-   good then remove the --dry option.
-
-
-=======================================================================
-Q. Migrating from Groupwise to Cyrus
-
-R. By Jamie Neil:
-
-I eventually managed to get the mail to migrate without errors using the 
-following options:
-
---sep1 /
-  - doesn't report separator so has to be set explicitly.
-
---nosyncacls
-  - doesn't support ACLs.
-
---skipheader '^Content-Type'
-  - MIME separator IDs seem to change every time a mail is accessed so 
-    this is required to stop duplicates.
-
---maxage 3650
-  - some messages just don't seem to want to transfer and produce the 
-    perl errors I mentioned before. This prevents the errors, but the
-    bad messages don't transfer.
-
-Even though the mail migrated OK, there are a couple of gotchas with 
-Groupwise IMAP:
-
-1) Some of the GW folders are not real folders and are not available
-to IMAP, the main problem one being "Sent Items". I could find no way
-of coping the contents of these folders. The nearest I got was to
-create a "real" folder and copy/move the sent items into it, but
-imapsync still didn't see the messages (I think because there is
-something funny about the reported dates/sizes).
-
-It think this problem has been rectified in GW6.5.
-
-2) The "skipheader '^Content-Type'" directive is required to stop 
-duplicate messages being created. GW seems to generate this field on
-the fly for messages that have MIME separators and so it's different
-every time.
-
-3) Version 6.0.1 of the Groupwise Internet Connector sucks. I was 
-getting server aborts when I pushed it a bit hard! I eventually had to
-upgrade to 6.0.4 which seems to be a lot more stable.
+Q. Migrating from or to the Cyrus imap server
 
+R. Read the dedicated 
+   https://imapsync.lamiral.info/FAQ.d/FAQ.Cyrus.txt
 
 =======================================================================
 Q. Migrating from iPlanet Messaging Server
@@ -142,37 +71,15 @@ I encounter many errors like this:
 
 R. GroupWise 7 seems buggy. Apply GroupWise 7 support pack 1
 
-=======================================================================
-Q. Migrating from David Tobit V10 (DvISE Mail Access Server MA-...)
-
-R. Use the following options:
-
-imapsync ... --prefix1 ""    --sep1 /  --idatefromheader ^
-             --nofoldersizes --useuid --nocheckmessageexists
 
 =======================================================================
-Q. Migrating from David Tobit V8
-   ("* OK IMAP4rev1 DvISE Mail Access Server MA-8.10a (0126)")
+Q. Migrating from or to David Tobit
 
-First try above V10 solution since improvments have been made
-to support Tobit.
-
-R. Use the following options :
-imapsync ... --prefix1 INBOX. --sep1 / --subscribe --subscribed 
+R. See FAQ.David_Tobit.txt
+   https://imapsync.lamiral.info/FAQ.d/FAQ.David_Tobit.txt
 
 =======================================================================
-Q. Migrating from Tobit David Server 6 
-   ("DvISE Mail Access Server MA-6.60a (0118)")
 
-First try above V10 solution since improvments have been made
-to support Tobit.
-
-R. Look at the discussion:
-http://www.linux-france.org/prj/imapsync_list/msg00582.html
-http://www.linux-france.org/prj/imapsync_list/threads.html#00582
-patch saved in ./patches/imapsync-1.337_tobit_V6.patch
-
-=======================================================================
 Q. I need to migrate 1250 mailboxes, passwords are in a MySQL Database.
 Can you tell me if your script suits my needs?
 
@@ -181,15 +88,6 @@ You have to extract users logins and passwords in a csv file.
 See the "HUGE MIGRATION" section in the README file.
 
 
-======================================================================
-Q: From MailEnable 1.75
-R: --sep1 "/" --prefix1 ""
-
-Q: From MailEnable 2.2
-R: --sep1 "." --prefix1 ""
-
-Q: To MailEnable
-R: --sep2 / --prefix2 "" --addheader --messageidnodomain --syncflagsaftercopy
 
 ======================================================================
 Q. From GMX IMAP4 StreamProxy
@@ -224,53 +122,6 @@ Q. From or to QQMail IMAP4Server
 
 R. imapsync ... --noabletosearch1
 
-======================================================================
-Q. From FirstClass to XXX
-   http://www.firstclass.com/
-
-R. Migrating from FirstClass is not easy because FirstClass, strangely, 
-   does not show all messages via IMAP. To make it show all messages,
-   a trick, painful to follow by hand, is moving emails 
-   out and back in, for each folder. May be it can be done by a script.
-
-  FirstClass releases prior to release 12 do not shows the "Sent" 
-  folder in IMAP but FirstClass release 12 shows it.
-  I advice you to upgrade to FirstClass release 12 before leaving it
-  with imapsync or another imap tool.
-
-Here is a command line used to migrate from FirtClass release 12:
-
-  imapsync ... \
-    --tmpdir /var/tmp --usecache \
-    --useheader Message-ID \
-    --idatefromheader \
-    --addheader \
-    --regextrans2 "s,(/|^) +,\$1,g" --regextrans2 "s, +(/|$),\$1,g" \
-    --regextrans2 "s/[\^]/_/g"       \
-    --regextrans2 "s/['\"\\\\]/_/g"  \
-    --regextrans2 "s,&AC8-,-,g" \
-    --regextrans2 "s,&APg-,oe,g" 
-
-On Windows:
-  imapsync.exe ... ^
-    --automap ^
-    --usecache ^
-    --useheader Message-ID ^
-    --idatefromheader ^
-    --addheader ^
-    --regextrans2 "s,(/|^) +,$1,g" ^
-    --regextrans2 "s, +(/|$),$1,g" ^
-    --regextrans2 "s/[\^]/_/g" ^
-    --regextrans2 "s/['\\]/_/g" ^
-    --regextrans2 "s,^&AC8-,-,g"  ^
-    --regextrans2 "s,^&APg-,oe,g"
-
-
-Special thanks to Kristian Wind and Joey Alexander for helping me
-writing this FAQ item. 
-See also this worth reading discussion in a Zimbra forum:
-http://www.zimbra.com/forums/migration/20349-help-needed-migrating-firstclass.html
-
 ======================================================================
 Q. From XXX to FTGate
 

FAQ.d/FAQ.Virus.txt

@@ -1,5 +1,5 @@
 
-$Id: FAQ.Virus.txt,v 1.5 2018/05/24 11:34:30 gilles Exp gilles $
+$Id: FAQ.Virus.txt,v 1.9 2020/01/21 21:35:53 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -14,17 +14,18 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Virus.txt
 Q. My virus scanner claims imapsync.exe is a virus/malware/trojan etc.
    What the hell?
 
-R1. Yes, I found the same. Two antivirus, Baidu and Jiangmin, report that 
-   imapsync might have a trojan in it. 
-   All others virus scanner say imapsync.exe is ok.
-   I've done this test on imapsync.exe release 1.727 on the two following
+R1. Yes, I find the same sometimes with few antivirus, some report that 
+   imapsync might have a trojan in it.
+   Most virus scanners say imapsync.exe and  is ok.
+   I've done this test on imapsync.exe releases 
+   1.977, 1.945, 1.836 on the two following
    meta-virus-scanners:
    https://www.virustotal.com/
    https://www.metadefender.com/
 
 R2. Explanation: 
     It may come from the fact that imapsync checks whether there is a
-    new realease available at http://imapsync.lamiral.info/VERSION
+    new release available at http://imapsync.lamiral.info/VERSION
     It's explained here:
     https://imapsync.lamiral.info/#NUMBERS
 
@@ -35,12 +36,39 @@ R3. There is no virus alerts reported for the Perl script imapsync
     itself. The binary imapsync.exe is just a compiled version of
     the script imapsync, with perl interpreter itself and all modules 
     needed by imapsync in order to make imapsync.exe a standalone software.
-	
-R4. imapsync.exe release 1.836 is reported as totally clean by both
+
+R4. Using online meta-virus-scanners
     https://www.virustotal.com/
+    https://www.metadefender.com/
+
+
+    Release 1.977
+    
+    imapsync.exe (64bit) release 1.977 is reported as not clean:
+    * https://www.virustotal.com/gui/file/8fd16a5eb2aa4d9a037a528654cc0ecdc63936955d289811dee31c6af474f4b0/detection
+      SecureAge APEX Malicious Jiangmin Backdoor.Kasidet.qt
+    
+    imapsync_32bit.exe release 1.977 is reported as totally clean:
+    https://www.virustotal.com/gui/url/94c89ef31190050bebc522f64e32d3f825ef0418cfc424c5847495b9857d57bf/detection
+
+    imapsync_bin_Darwin release 1.977 is reported as totally clean:
+    * https://www.virustotal.com/gui/file/41f107f4f7c86b15c392aef3c69a7f285f3b2a5f02328a1342903eddc5a5a06f/details
+    * https://metadefender.opswat.com/results#!/file/bzIwMDEwNUJKb2dXa1N3eWVJSDFoZ1d5clBrZ1U/regular/overview
+
+    Release 1.945
+    
+    imapsync.exe release 1.945 is reported as totally clean:
+    * https://www.virustotal.com/gui/file/c122fd3bc82c9c784e8b27504e1843f95b1dfd9beba05f50a2893853ad4a6c59/details
+    * https://metadefender.opswat.com/results#!/file/bzE5MTIwNVN5ZXNDa2NMVEhCeWJpQTFjVVRT/regular/overview
+
+    imapsync_64bit.exe release 1.945 is reported as totally clean:
+    * https://www.virustotal.com/gui/file/f561c6e457c3173f7b0379ee8cc267347feff1bc1991cf47fea7c513c81ccb0a/detection
+    * https://metadefender.opswat.com/results#!/file/bzE5MDkzMHJ5dk9qb2gxZEhIMXVkaWloeV9C/regular/multiscan
+
+
+    Release 1.836
+    
+    imapsync.exe release 1.836 is reported as totally clean:
     https://www.virustotal.com/#/file/2a33eb93198da77677a4013a5dfb4868cb6aa8f4513e2102a55884b258d19043/detection
-    
-    and https://metadefender.opswat.com/
     https://metadefender.opswat.com/results#!/file/ZTE4MDMxN1MxZTk4aGhxdEdCSlpxVTJoOVlN/regular/information
-    
-    
+

FAQ.d/FAQ.XOAUTH2.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.XOAUTH2.txt,v 1.15 2018/08/28 21:48:53 gilles Exp gilles $
+$Id: FAQ.XOAUTH2.txt,v 1.18 2022/04/05 13:48:52 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -10,9 +10,47 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.XOAUTH2.txt
   Imapsync tips to use XOAUTH2 authentication (Gmail) and old XOAUTH 
 =======================================================================
 
+Questions answered in this FAQ are:
+
+Q. Is XOAUTH2 authentication available with imapsync to authenticate
+   my personnal gmail account? (Quick answer: not yet)
+
+Q. Is XOAUTH2 authentication available with imapsync to authenticate
+   my personnal Office365 account? (Quick answer: not yet)
+
+Q. Is XOAUTH2 authentication available with imapsync to globally 
+   authenticate gmail users, ie as an admin?
+
+Q. Imapsync XOAUTH2 fails with the following message, how to fix that?
+
+Q. How to use XOAUTH2 via a json file to globally authenticate gmail users? 
+
+Q. How to use XOAUTH2 via pk12 file to globally authenticate gmail users?
+
+Q. How to use a proxy with XOAUTH2 authentication?
+
+Q. How to use old XOAUTH to globally authenticate gmail users?
+
+
+Now the questions again with their answers.
 
 =======================================================================
-Q. Is XOAUTH2 authentication available with imapsync?
+Q. Is XOAUTH2 authentication available with imapsync to authenticate
+   my personnal gmail account? (Quick answer: not yet)
+
+R. No. But I'm working on that. 
+   XOAUTH2 is a mess or at least, each time I look at it, it leaves 
+   my brain like a mess.
+
+=======================================================================
+Q. Is XOAUTH2 authentication available with imapsync to authenticate
+   my personnal Office365 account? (Quick answer: not yet)
+
+R. No. But I'm working on that too.
+
+=======================================================================
+Q. Is XOAUTH2 authentication available with imapsync to globally 
+   authenticate gmail users, ie as an admin?
 
 R. Yes, but XOAUTH2 has been really tested on Unix systems, 
    less profund on Windows but it should work.
@@ -97,7 +135,7 @@ https://developers.google.com/identity/protocols/OAuth2
 
 
 =======================================================================
-Q. How to use XOAUTH2 via pk12 file to globally authenticate gmail users? 
+Q. How to use XOAUTH2 via pk12 file to globally authenticate gmail users?
 
 R. First, consider the XOAUTH2 feature at a prototype level.
 
@@ -116,7 +154,7 @@ also called cpanminus.
 
 The code and first explanation comes from Joaquin Lopez at
 https://github.com/imapsync/imapsync/pull/25
-http://www.linux-france.org/prj/imapsync_list/msg02129.html
+http://linux-france.tk/prj/imapsync_list/msg02129.html
 
 Also, the binary command "openssl" is needed since it is used to 
 convert the pk12 file.
@@ -154,7 +192,8 @@ imapsync \
 
 Use your own xoauth2 values.
 
-
+See also
+http://www.notearthday.org/nedtech/2016/05/creating-creating-oauth2-credentials-with-google-apps/
 
 =======================================================================
 Q. How to use a proxy with XOAUTH2 authentication?
@@ -171,7 +210,7 @@ environment variable, if it isn't already set. Example:
 
 
 =======================================================================
-Q. How to use XOAUTH to globally authenticate gmail users?
+Q. How to use old XOAUTH to globally authenticate gmail users?
 
 R0. XOAUTH is considered obsolete and superseded by XOAUTH2
     Anyway the manage part might be the same (I don't know).

FAQ.d/FAQ.Yahoo.txt

@@ -1,5 +1,5 @@
 
-$Id: FAQ.Yahoo.txt,v 1.8 2018/05/24 11:34:30 gilles Exp gilles $
+$Id: FAQ.Yahoo.txt,v 1.15 2022/05/05 14:17:22 gilles Exp gilles $
 
 This document is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -10,39 +10,28 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Yahoo.txt
                      Imapsync tips for Yahoo.
 =======================================================================
 
+Questions answered in this FAQ are:
+
+Q. Synchronizing from Yahoo to XXX
+
+Q. Synchronizing from XXX to Yahoo
+
+Now the questions again with their answers.
 
 =======================================================================
 Q. Synchronizing from Yahoo to XXX
 
-R. Use --host1 imap.mail.yahoo.com --ssl1
+R. Use --host1 imap.mail.yahoo.com 
 
-./imapsync \
+Example:
+
+  ./imapsync \
     --host1 imap.mail.yahoo.com \
-    --user1 billy \
-    --password1 secret \
-    --ssl1 \
+    --user1 Billy@yahoo.com \
+    --password1 billysecreta \
     --host2 XXX \
-    --user2 billy \
-    --password2 secret \
-    --noabletosearch --errorsmax 10000 
-
-SSL is mandatory for yahoo since November 2011.
-
-Why increasing --errorsmax to 10000?
-Because during the sync you may encounter several lines like this one:
-- msg Inbox/12600 {0} S[36012] F[] I[27-Nov-2015 07:08:35 +0000] could not be fetched: 190 NO [UNAVAILABLE] UID FETCH Server error while fetching messages
-- msg Inbox/12600 skipped.
-I don't know why it happens, why Yahoo server says 
-"NO [UNAVAILABLE] UID FETCH Server error while fetching messages"
-Usually another run syncs the message successfully.
-
-Why --noabletosearch option?
-Yahoo IMAP does not honor well the SEARCH imap command (Dec 2017).
-So don't use the --search option.
-You can use --minage or/and --maxage but then add also --noabletosearch
-
-Yahoo bandwidth per connexion is relatively low, 50 KiB/s to 500 KiB/s max, usually 100 KiB/s.
-Consider using --useuid to speed up large accounts.
+    --user2 Billy@other.com \
+    --password2 billysecretb
 
 
 You also need to go to Yahoo, security and enable 
@@ -70,6 +59,47 @@ Another solution:
     since it is not in the predefined apps.
   * Use this password with imapsync.
 
+=======================================================================
+Q. Synchronizing from XXX to Yahoo
+
+R. Example:
+
+On Linux:
+  ./imapsync \
+    --host1 XXX \
+    --user1 Billy@other.com \
+    --password1 billysecreta \
+    --host2 imap.mail.yahoo.com \
+    --user2 Billy@yahoo.com \
+    --password2 billysecretb \
+    --regexmess 's{(?<![\n])\z}{\r\n}gxms'
+
+On Windows:
+  imapsync.exe \
+    --host1 XXX \
+    --user1 Billy@other.com \
+    --password1 billysecreta \
+    --host2 imap.mail.yahoo.com \
+    --user2 Billy@yahoo.com \
+    --password2 billysecretb \
+    --regexmess "s{(?<![\n])\z}{\r\n}gxms"
+
+
+
+You wonder why there is this ugly regex, what it does and where does it 
+come from?
+
+It's there to get rid of the errors 
+"BAD [CLIENTBUG] Additional arguments found after last expected argument"
+when copying some messages to Yahoo.
+It adds a final \r\n on a message if a final \r\n is missing.
+See https://github.com/imapsync/imapsync/issues/209
+
+https://imapsync.lamiral.info/README
+...
+ --regexmess    reg  : Apply the whole regex to each message before transfer.
+                       Example: 's/\000/ /g' # to replace null by space.
+ --regexmess    reg  : and this one, etc.
 
 =======================================================================
 =======================================================================

FAQ.d/FAQ.Yandex.txt

@@ -0,0 +1,26 @@
+#!/bin/cat
+$Id: FAQ.Yandex.txt,v 1.1 2019/09/12 10:08:05 gilles Exp gilles $
+
+This document is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.Kerio.txt
+
+=======================================================================
+              Imapsync tips for Yandex accounts.
+=======================================================================
+
+
+
+
+=======================================================================
+Q. How to migrate from or to Yandex
+
+
+R. Use:
+
+  imapsync ... --useheader Message-Id --noabletosearch
+
+The option --noabletosearch is needed only if options --maxage or --minage
+are used. I don't know which --search commands could work, maybe none.
+
+The imap server for --host1 or --host2 is usually imap.yandex.ru

FAQ.d/FAQ.Zimbra.txt

@@ -1,5 +1,5 @@
 #!/bin/cat
-$Id: FAQ.Zimbra.txt,v 1.4 2019/01/28 22:39:28 gilles Exp gilles $
+$Id: FAQ.Zimbra.txt,v 1.11 2021/04/18 09:15:45 gilles Exp gilles $
 
 This documentation is also available online at
 https://imapsync.lamiral.info/FAQ.d/
@@ -15,6 +15,29 @@ https://wiki.zimbra.com/wiki/Guide_to_imapsync
 It's obsolete and it will give you, and me, more difficulties than 
 doing a standard sync without additional options.
 
+Questions answered in this FAQ are:
+
+Q. How to migrate from Zimbra to XXX?
+
+Q. How to migrate from Zimbra with an admin account?
+
+Q. I got this "NO APPEND failed" error:
+could not append ( Subject:... Flags:[\Flagged \Seen EmailswithStickyNotes:FLAG51135] ) 
+to folder XXX: 316 NO APPEND failed
+What can I do?
+
+Q. Zimbra disconnects without reason
+   Could not select: * BYE ... Zimbra IMAP4rev1 server closing connection
+
+
+Now the questions again with their answers.
+
+
+=======================================================================
+Q. How to migrate from Zimbra to XXX?
+
+The basic default (only credendials) should work fine with Zimbra.
+
 
 =======================================================================
 Q. How to migrate from Zimbra with an admin account?
@@ -28,5 +51,33 @@ https://zimbra.github.io/adminguide/8.8.9/index.html#_administrator_accounts
 
 Thanks to Richard Street from thinkround for this tip.
 
+=======================================================================
+Q. I got this "NO APPEND failed" error:
+could not append ( Subject:... Flags:[\Flagged \Seen EmailswithStickyNotes:FLAG51135] ) 
+to folder XXX: 316 NO APPEND failed
+What can I do?
+
+R1.
+https://zimbra.github.io/adminguide/8.8.10/index.html
+Flagging
+Users can create flags and assign them to messages, contacts, and files in Briefcase folders.
+In the Admin Console, see COS -> Features -> Mail Features container
+
+Is Flagging on on host2? 
+If yes, see https://imapsync.lamiral.info/FAQ.d/FAQ.Flags.txt
+
+=======================================================================
+Q. Zimbra disconnects without reason
+   Could not select: * BYE ... Zimbra IMAP4rev1 server closing connection
+
+R. The cause of this problem may be that some users have a subscription 
+   of a shared folder from a mailbox locked and so the shared folder.
+   The access problems may happen in a lot of folders, 
+   not only the shared/locked one.
+
+   It should work after removing the locked/shared folder.
+
+   Thanks to Danny Herrlitz for this input!
+
 =======================================================================
 =======================================================================

FAQ.d/FAQ.iCloud.txt

@@ -0,0 +1,24 @@
+#!/bin/cat
+# $Id: FAQ.iCloud.txt,v 1.1 2022/04/13 11:44:19 gilles Exp gilles $
+
+This documentation is also available online at
+https://imapsync.lamiral.info/FAQ.d/
+https://imapsync.lamiral.info/FAQ.d/FAQ.iCloud.txt
+
+
+=======================================================================
+               Imapsync tips for iCloud accounts.
+=======================================================================
+
+
+Q. How to migrate from or to an iCloud account?
+
+R. From https://support.apple.com/en-us/HT202304
+
+Use "imap.mail.me.com" as the imap server and use an app password.
+
+How to generate an app password is described at
+https://support.apple.com/kb/HT204397
+
+It's not necessary to precise the port and the ssl thing, imapsync
+auto-detect this.

FAQ.d/htaccess.txt

@@ -1,53 +1,82 @@
 
-# $Id: htaccess.txt,v 1.23 2019/02/16 22:42:25 gilles Exp gilles $
+# $Id: htaccess.txt,v 1.33 2022/04/06 10:27:52 gilles Exp gilles $
 
 
-AddDescription "<b>Back</b> to Imapsync main page."                ..
-AddDescription "<b>APPEND errors</b>."                             FAQ.Admin_Authentication.txt
-AddDescription "<b>Authenticate via an admin account</b>."         FAQ.APPEND_errors.txt
-AddDescription "<b>Authentication failures</b>."                   FAQ.Authentication_failure.txt
-AddDescription "<b>Archiving</b> tips."                            FAQ.Archiving.txt
-AddDescription "<b>Big mailboxes</b> tips."                        FAQ.Big_Mailbox.txt
-AddDescription "<b>Connection issues</b>."                         FAQ.Connection.txt
-AddDescription "<b>Contacts & Calendars & Chat issues</b>."        FAQ.Contacts_Calendars.txt
-AddDescription "<b>Dates issues</b>."                              FAQ.Dates.txt
-AddDescription "<b>Domino</b>."                                    FAQ.Domino.txt
-AddDescription "<b>Dovecot</b>."                                   FAQ.Dovecot.txt
-AddDescription "<b>Duplicated</b> messages issues."                FAQ.Duplicates.txt
-AddDescription "<b>Emptying</b> an account."                       FAQ.Emptying.txt
-AddDescription "<b>Exchange 20xx</b> and <b>Office365</b>."        FAQ.Exchange.txt
-AddDescription "<b>Changing folders names</b>."                    FAQ.Folders_Mapping.txt
-AddDescription "<b>Selecting folders</b>."                         FAQ.Folders_Selection.txt
-AddDescription "<b>Folders sizes with Imapsync</b>."               FAQ.Folders_Sizes.txt
+AddDescription "<b>Back</b> to Imapsync main page."                     ..
+AddDescription "<b>Where this Description column comes from</b>."       htaccess.txt
+AddDescription "<b>Private</b>."                                        memo
+
+AddDescription "<b>APPEND errors</b>."                                  FAQ.Admin_Authentication.txt
+AddDescription "<b>Authenticate via an admin account</b>."              FAQ.APPEND_errors.txt
+AddDescription "<b>Archiving</b> tips."                                 FAQ.Archiving.txt
+AddDescription "<b>Authentication failures</b>."                        FAQ.Authentication_failure.txt
+
+AddDescription "<b>Bandwidth questions</b> tips."                       FAQ.Bandwidth.txt
+AddDescription "<b>Big mailboxes</b> tips."                             FAQ.Big_Mailbox.txt
+
+AddDescription "<b>Changing folders names</b>."                         FAQ.Folders_Mapping.txt
+AddDescription "<b>Connection issues</b>."                              FAQ.Connection.txt
+AddDescription "<b>Contacts & Calendars & Chat issues</b>."             FAQ.Contacts_Calendars.txt
+AddDescription "<b>Cyrus</b> accounts."                                 FAQ.Cyrus.txt
 
 
-AddDescription "<b>Flags</b>."                                     FAQ.Flags.txt
-AddDescription "<b>General</b> and <b>pot-pourri</b> issues"       FAQ.General.txt
-AddDescription "<b>Gmail</b> accounts."                            FAQ.Gmail.txt
-AddDescription "<b>ISP tips</b>."                                  FAQ.ISP.txt
-AddDescription "<b>Massive/bulk migrations</b>."                   FAQ.Massive.txt
-AddDescription "<b>Memory issues</b>."                             FAQ.Memory.txt
-AddDescription "<b>Migration Plan</b>."                            FAQ.Migration_Plan.txt
-AddDescription "<b>The Online UI service Q&R</b>."                 FAQ.OnlineUI.txt
-AddDescription "<b>Password & special characters on Windows</b>."  FAQ.Passwords_on_Windows.txt
-AddDescription "<b>Password & special characters on Unix</b>."     FAQ.Passwords_on_Unix.txt
-AddDescription "<b>POP3 issues</b>."                               FAQ.POP3.txt
-AddDescription "<b>Selecting messages</b>."                        FAQ.Messages_Selection.txt
-AddDescription "<b>Oracle-UCS</b>."                                FAQ.Oracle-UCS.txt
-AddDescription "<b>Checklist before release a new release</b>."    FAQ.Release_Checklist.txt
-AddDescription "<b>Guidelines to report bugs</b>."                 FAQ.Reporting_Bugs.txt
-AddDescription "<b>Security</b>."                                  FAQ.Security.txt
-AddDescription "<b>SSL errors</b>."                                FAQ.SSL_errors.txt
-AddDescription "<b>SmarterMail</b>."                               FAQ.SmarterMail.txt
+AddDescription "<b>Dates issues</b>."                                   FAQ.Dates.txt
+AddDescription "<b>David Tobit</b> accounts."                           FAQ.David_Tobit.txt
+AddDescription "<b>DBmail</b> accounts."                                FAQ.DBmail.txt
+AddDescription "<b>Docker</b> usage."                                   FAQ.Docker.txt
+AddDescription "<b>Domino</b> accounts."                                FAQ.Domino.txt
+AddDescription "<b>Dovecot</b> accounts."                               FAQ.Dovecot.txt
+AddDescription "<b>Duplicated</b> messages issues."                     FAQ.Duplicates.txt
+AddDescription "<b>Emptying</b> an account."                            FAQ.Emptying.txt
+AddDescription "<b>Exchange 20xx</b> and <b>Office365</b> accounts."    FAQ.Exchange.txt
+AddDescription "<b>Selecting folders</b>."                              FAQ.Folders_Selection.txt
+AddDescription "<b>Folders sizes with Imapsync</b>."                    FAQ.Folders_Sizes.txt
 
-AddDescription "<b>TTL (Time To Live) delay in DNS configuration</b>." FAQ.TTL.txt
-AddDescription "<b>Option --usecache and inodes</b>."              FAQ.Use_cache.txt
-AddDescription "<b>User concurrent access</b>."                    FAQ.User_Concurrent_Access.txt
-AddDescription "<b>Virus scanners on Imapsync</b>."                FAQ.Virus.txt
-AddDescription "<b>Why use --addheader?</b>."                      FAQ.Use_addheader.txt
-AddDescription "<b>Various imap software servers</b>."             FAQ.Various_Software_Servers.txt
-AddDescription "<b>XOAUTH2</b> (<b>Gmail</b>)."                    FAQ.XOAUTH2.txt
-AddDescription "<b>Yahoo</b>."                                     FAQ.Yahoo.txt
-AddDescription "<b>Zimbra</b>."                                    FAQ.Zimbra.txt
-AddDescription "<b>Where this Description column comes from</b>."  htaccess.txt
+
+
+
+AddDescription "<b>FirstClass</b> accounts."                            FAQ.FirstClass.txt
+AddDescription "<b>Flags tips and issues</b>."                          FAQ.Flags.txt
+AddDescription "<b>General Data Protection Regulation</b> (GDPR)."      FAQ.GDPR.txt
+AddDescription "<b>General</b> and <b>pot-pourri</b> issues"            FAQ.General.txt
+AddDescription "<b>Gmail</b> accounts."                                 FAQ.Gmail.txt
+AddDescription "<b>GoDaddy</b> accounts."                               FAQ.GoDaddy.txt
+AddDescription "<b>IceWarp</b> accounts."                               FAQ.IceWarp.txt
+AddDescription "<b>ISP tips</b>."                                       FAQ.ISP.txt
+AddDescription "<b>James</b> accounts."                                 FAQ.James.txt
+AddDescription "<b>Kerio</b> accounts."                                 FAQ.Kerio.txt
+AddDescription "<b>MailEnable</b> accounts."                            FAQ.MailEnable.txt
+AddDescription "<b>Massive and bulk</b> migrations."                    FAQ.Massive.txt
+AddDescription "<b>Log file, content, errors and tips</b>."             FAQ.Logfile.txt
+AddDescription "<b>Memory</b> issues."                                  FAQ.Memory.txt
+AddDescription "Migration <b>Plan</b>."                                 FAQ.Migration_Plan.txt
+AddDescription "<b>Office365</b> accounts."                             FAQ.Office365.txt
+AddDescription "<b>The Online UI service Q&R</b>."                      FAQ.OnlineUI.txt
+AddDescription "<b>Password & special characters on Windows</b>."       FAQ.Passwords_on_Windows.txt
+AddDescription "<b>Password & special characters on Unix</b>."          FAQ.Passwords_on_Unix.txt
+AddDescription "<b>Password & special characters on Mac OS X</b>."      FAQ.Passwords_on_Mac.txt
+AddDescription "<b>POP3 issues</b>."                                    FAQ.POP3.txt
+AddDescription "<b>Principles & design decisions</b>."                  FAQ.Principles.txt
+AddDescription "<b>Selecting messages</b>."                             FAQ.Messages_Selection.txt
+AddDescription "<b>How to transfer too big messages</b>."               FAQ.Messages_Too_Big.txt
+AddDescription "Why the imapsync web site <b>looks so old?</b>"         FAQ.Old_Style_Web_Design.txt
+AddDescription "<b>UCS</b> accounts."                                   FAQ.UCS.txt
+
+AddDescription "<b>Checklist before release a new release</b>."         FAQ.Release_Checklist.txt
+AddDescription "<b>Guidelines to report bugs</b>."                      FAQ.Reporting_Bugs.txt
+AddDescription "<b>Security</b>."                                       FAQ.Security.txt
+AddDescription "<b>SSL errors</b>."                                     FAQ.SSL_errors.txt
+AddDescription "<b>SmarterMail</b> accounts."                           FAQ.SmarterMail.txt
+
+AddDescription "<b>TTL (Time To Live) delay in DNS configuration</b>."  FAQ.TTL.txt
+AddDescription "<b>Why Imapsync can't do two ways syncs</b>."           FAQ.Two_Ways_Sync.txt
+AddDescription "<b>Option --usecache and inodes</b>."                   FAQ.Use_cache.txt
+AddDescription "<b>User concurrent access</b>."                         FAQ.User_Concurrent_Access.txt
+AddDescription "<b>Virus scanners on Imapsync</b>."                     FAQ.Virus.txt
+AddDescription "<b>Why use --addheader?</b>."                           FAQ.Use_addheader.txt
+AddDescription "<b>Various imap software servers</b>."                  FAQ.Various_Software_Servers.txt
+AddDescription "<b>XOAUTH2</b> (<b>Gmail</b>)."                         FAQ.XOAUTH2.txt
+AddDescription "<b>Yahoo</b> accounts."                                 FAQ.Yahoo.txt
+AddDescription "<b>Yandex</b> accounts."                                FAQ.Yandex.txt
+AddDescription "<b>Zimbra</b> accounts."                                FAQ.Zimbra.txt
 

INSTALL.d/Dockerfile

@@ -1,35 +1,61 @@
 ## Dockerfile for building a docker imapsync image
 
-# $Id: Dockerfile,v 1.14 2018/09/16 10:42:11 gilles Exp gilles $
+# $Id: Dockerfile,v 1.43 2022/05/17 14:29:36 gilles Exp gilles $
 # I use the following command to build the image:
 #
 #  docker build -t gilleslamiral/imapsync . 
 #
 # where this Dockerfile is in the current directory
 # 
+# This Dockerfile build an image with two imapsync
+# 1) One comes from https://imapsync.lamiral.info/imapsync
+#    and goes to /usr/bin/imapsync in the Docker image
+#    It is used with the command:
+#
+#    docker run gilleslamiral/imapsync imapsync ...
+# 
+# 2) One comes from the local file ./imapsync
+#    and goes to /imapsync in the Docker image
+#    It is used with the command:
+#
+#    docker run gilleslamiral/imapsync /imapsync ...
+
+
+# I thank you very much
 # I like thanks
 # I like stars
-# I also like (and need) money
-# I thank you very much in advance
+# I like money to keep on doing this stuff
 
+# Number of imapsync images pulled so far (2022_04_04): 673551
+# Command used:
+# curl -s https://hub.docker.com/v2/repositories/gilleslamiral/imapsync/ | jq '.pull_count'
 
-FROM debian:stretch 
+# Debian Bullseye is Debian 11
 
-LABEL maintainer "gilles@lamiral.info"
+FROM debian:bullseye 
 
-# Put a copy of the Dockerfile in the image itself
-# It can help future maintenance, isn't it?
+LABEL   maintainer="Gilles LAMIRAL <gilles@lamiral.info>" \
+        description="Imapsync" \
+        documentation="https://imapsync.lamiral.info/#doc"
 
-COPY Dockerfile /
+# I put a copy of the Dockerfile in the image itself
+# It can help maintenance, isn't it?
+# Also put optionally my local and usually more recent imapsync on /, for testing purpose
 
-RUN apt-get update \
+COPY Dockerfile imapsyn[c] prerequisites_imapsyn[c] /
+
+RUN set -xe && \
+  apt-get update \
   && apt-get install -y \
-  libjson-webtoken-perl \
   libauthen-ntlm-perl \
   libcgi-pm-perl \
   libcrypt-openssl-rsa-perl \
+  libcrypt-openssl-pkcs12-perl \
   libdata-uniqid-perl \
+  libencode-imaputf7-perl \
   libfile-copy-recursive-perl \
+  libfile-tail-perl \
+  libio-compress-perl \
   libio-socket-ssl-perl \
   libio-socket-inet6-perl \
   libio-tee-perl \
@@ -39,6 +65,7 @@ RUN apt-get update \
   libparse-recdescent-perl \
   libmodule-scandeps-perl \
   libpar-packer-perl \
+  libproc-processtable-perl \
   libreadonly-perl \
   libregexp-common-perl \
   libsys-meminfo-perl \
@@ -52,17 +79,33 @@ RUN apt-get update \
   wget \
   make \
   cpanminus \
-  && rm -rf /var/lib/apt/lists/*
+  lsof \
+  ncat \
+  openssl \
+  ca-certificates \
+  && rm -rf /var/lib/apt/lists/* \
+  && cpanm IO::Socket::SSL
 
-RUN wget -N https://imapsync.lamiral.info/imapsync \
-  https://imapsync.lamiral.info/prerequisites_imapsync \
-  && cp imapsync /usr/bin/imapsync \
-  && chmod +x /usr/bin/imapsync # just_a_comment_to_force_update 2018_09_13_14_44_03
+# I added the IO::Socket::SSL update to avoid the annoying, confusing and useless warning
+# DEBUG: .../IO/Socket/SSL.pm:1177: global error: Undefined SSL object
 
-USER nobody
+RUN set -xe \
+  && cd /usr/bin/ \
+  && pwd \
+  && wget -N --no-check-certificate https://imapsync.lamiral.info/imapsync \
+        https://imapsync.lamiral.info/prerequisites_imapsync \
+        https://raw.githubusercontent.com/google/gmail-oauth2-tools/master/python/oauth2.py \
+  && chmod +x imapsync oauth2.py \
+  && /usr/bin/imapsync --testslive && /usr/bin/imapsync --tests # just_a_comment_to_force_update 2022_04_04_21_16_50
+
+USER nobody:nogroup
 
 ENV HOME /var/tmp/
 
+WORKDIR /var/tmp/
+
+STOPSIGNAL SIGINT
+
 CMD ["/usr/bin/imapsync"]
 
 # 

INSTALL.d/INSTALL.ANY.txt

@@ -1,4 +1,4 @@
-# $Id: INSTALL.ANY.txt,v 1.58 2018/05/24 11:43:31 gilles Exp gilles $
+# $Id: INSTALL.ANY.txt,v 1.62 2020/06/03 07:11:10 gilles Exp gilles $
 # 
 # This is the main INSTALL file for imapsync.
 # imapsync : IMAP sync and migrate tool.
@@ -13,7 +13,8 @@ https://imapsync.lamiral.info/INSTALL.d/INSTALL.ANY.txt
 
  imapsync works fine under any operating system with Perl and Perl modules (listed below).
  imapsync.exe works fine standalone under Windows XP, Vista, Seven, 20XX, either 32 or 64bit.
-
+ imapsync_64bit.exe works fine standalone under Seven, 20XX, but only 64bit.
+ 
 =======================================================================
                Installing imapsync on WINDOWS 
 =======================================================================
@@ -37,9 +38,15 @@ Those documents are also available at https://imapsync.lamiral.info/INSTALL.d/
 - Debian
 - Ubuntu
 - AWS EC2
+- Arch Linux
+- Docker
 - Online UI like /X
 
-If you are not on one of these systems then read the section
+If you are not familiar with any of these systems but you have to choose
+one to install imapsync, imapsync online, or the latest imapsync then 
+Centos 7 (not 8 for now) is the easiest system to consider (June 2020).
+
+If you are not on one of these systems then read the last section
 below called "Installing imapsync on other Unixes".
 
 =======================================================================
@@ -47,6 +54,7 @@ below called "Installing imapsync on other Unixes".
 =======================================================================
 
 Easy.
+
 Read the file INSTALL.d/INSTALL.Darwin.txt
 This document is also available at
 https://imapsync.lamiral.info/INSTALL.d/INSTALL.Darwin.txt
@@ -56,6 +64,7 @@ https://imapsync.lamiral.info/INSTALL.d/INSTALL.Darwin.txt
 =======================================================================
 
 Easy.
+
 Read the file INSTALL.d/INSTALL.FreeBSD.txt
 This document is also available at
 https://imapsync.lamiral.info/INSTALL.d/INSTALL.FreeBSD.txt
@@ -66,6 +75,7 @@ https://imapsync.lamiral.info/INSTALL.d/INSTALL.FreeBSD.txt
 =======================================================================
 
 Easy.
+
 Read the file INSTALL.d/INSTALL.Centos.txt
 This document is also available at
 https://imapsync.lamiral.info/INSTALL.d/INSTALL.Centos.txt
@@ -76,42 +86,58 @@ https://imapsync.lamiral.info/INSTALL.d/INSTALL.Centos.txt
 =======================================================================
 
 Easy.
+
 Read the file INSTALL.d/INSTALL.CPanel.txt
 This document is also available at
 https://imapsync.lamiral.info/INSTALL.d/INSTALL.CPanel.txt
 
 
 =======================================================================
-               Installing imapsync on Debian 6 or 7 
+               Installing imapsync on Debian
 =======================================================================
 
 Not so easy.
+
 See the file INSTALL.d/INSTALL.Debian.txt
 This document is also available at
 https://imapsync.lamiral.info/INSTALL.d/INSTALL.Debian.txt
 
 =======================================================================
-               Installing imapsync on Ubuntu 12 or 14 
+               Installing imapsync on Ubuntu
 =======================================================================
 
 Not so easy.
+
 See the file INSTALL.d/INSTALL.Ubuntu.txt
 This document is also available at
 https://imapsync.lamiral.info/INSTALL.d/INSTALL.Ubuntu.txt
 
 =======================================================================
-               Installing imapsync on AWS EC2 
+               Installing imapsync on AWS 
 =======================================================================
 
 Not so easy.
+
 See the file INSTALL.d/INSTALL.AWS_EC2.txt
 This document is also available at
 https://imapsync.lamiral.info/INSTALL.d/INSTALL.AWS_EC2.txt
 
+=======================================================================
+               Installing imapsync on ArchLinux
+=======================================================================
+
+Not so easy.
+
+See the file INSTALL.d/INSTALL.ArchLinux.txt
+This document is also available at
+https://imapsync.lamiral.info/INSTALL.d/INSTALL.ArchLinux.txt
+
 =======================================================================
                Installing imapsync online like /X 
 =======================================================================
 
+Not easy.
+
 Follow the document INSTALL.d/INSTALL.OnlineUI.txt
 This document is also available at
 https://imapsync.lamiral.info/INSTALL.d/INSTALL.OnlineUI.txt
@@ -121,6 +147,7 @@ https://imapsync.lamiral.info/INSTALL.d/INSTALL.OnlineUI.txt
                Installing imapsync on other Unixes
 =======================================================================
 
+Not easy.
 
 Purchase imapsync at
 https://imapsync.lamiral.info/
@@ -163,7 +190,8 @@ Install make in case it is not already installed.
 Run the "cpanm" command with the missing Perl modules as arguments.
 For example it can be:
 
-  cpanm Authen::NTLM Data::Uniqid File::Copy::Recursive IO::Tee Mail::IMAPClient Unicode::String
+  cpanm Authen::NTLM Data::Uniqid File::Copy::Recursive IO::Tee \
+        Mail::IMAPClient Unicode::String Encode::IMAPUTF7
 
 Once you've run the "cpanm" command, you can rerun "sh prerequisites_imapsync" 
 to verify everything is ok:

INSTALL.d/INSTALL.AWS_EC2.txt

@@ -1,5 +1,5 @@
 #!/bin/cat 
-# $Id: INSTALL.AWS_EC2.txt,v 1.5 2018/05/24 11:43:31 gilles Exp gilles $
+# $Id: INSTALL.AWS_EC2.txt,v 1.9 2022/05/17 14:32:47 gilles Exp gilles $
 
 This documentation is also located online at 
 https://imapsync.lamiral.info/INSTALL.d/
@@ -9,48 +9,101 @@ https://imapsync.lamiral.info/INSTALL.d/INSTALL.AWS_EC2.txt
                Installing imapsync on AWS EC2
 =======================================================================
 
+
 AWS: Amazon Web Service
 
+A) cat /etc/system-release
+Amazon Linux release 2 (Karoo)
+
+  sudo yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
+  sudo yum install imapsync
+
+  imapsync --version
+  1.727
+
+To install a more recent imapsync
+
+  sudo yum install perl-App-cpanminus \
+ perl-Dist-CheckConflicts \
+ perl-HTML-Parser \
+ perl-libwww-perl \
+ perl-Module-Implementation \
+ perl-Module-ScanDeps \
+ perl-Package-Stash \
+ perl-Package-Stash-XS \
+ perl-PAR-Packer \
+ perl-Regexp-Common \
+ perl-Sys-MemInfo \
+ perl-Test-Fatal \
+ perl-Test-Mock-Guard \
+ perl-Test-Requires \
+ perl-Test-Deep \
+ perl-File-Tail \
+ perl-Unicode-String \
+ perl-Test-NoWarnings \
+ perl-Test-Simple \
+ perl-Test-Warn \
+ perl-Sub-Uplevel \
+ perl-Proc-ProcessTable
+
+  sudo cpanm Encode::IMAPUTF7
+  wget -N https://imapsync.lamiral.info/imapsync
+  chmod +x imapsync
+  ./imapsync --testslive
+
+  ./imapsync --version # command typed on 14 April 2020
+  1.983
+
+  sudo cpanm CGI 
+
+
+B) Old Amazon Linux AMI release 2017.03
+
 cat /etc/system-release
 Amazon Linux AMI release 2017.03
 
 
 sudo yum install  \
-perl-Class-Load \
-perl-IO-Compress \
-perl-Crypt-OpenSSL-RSA \
-perl-Data-Dumper \
+perl-Class-Load   \
+perl-IO-Compress  \
+perl-Crypt-OpenSSL-RSA   \
+perl-Data-Dumper         \
 perl-Dist-CheckConflicts \
+perl-ExtUtils-Embed      \
 perl-File-Copy-Recursive \
+perl-File-Tail       \
 perl-IO-Socket-INET6 \
-perl-IO-Socket-SSL \
-perl-JSON \
-perl-HTML-Parser \
-perl-libwww-perl \
+perl-IO-Socket-SSL   \
+perl-JSON            \
+perl-HTML-Parser     \
+perl-libwww-perl     \
 perl-Mail-IMAPClient \
 perl-Module-Implementation \
-perl-Module-Runtime \
-perl-Module-ScanDeps \
-perl-Net-SSLeay \
-perl-Package-Stash \
+perl-Module-Runtime   \
+perl-Module-ScanDeps  \
+perl-Net-SSLeay       \
+perl-Package-Stash    \
 perl-Package-Stash-XS \
 perl-Parse-RecDescent \
-perl-Readonly \
-perl-Regexp-Common \
-perl-Sys-MemInfo \
-perl-TermReadKey \
-perl-Test-Fatal \
-perl-Test-MockObject \
-perl-Test-Simple \
-perl-Test-Pod \
-perl-Test-Requires \
-perl-Try-Tiny \
-perl-Unicode-String \
-perl-URI \
-perl-ExtUtils-Embed \
+perl-Readonly         \
+perl-Regexp-Common    \
+perl-Sys-MemInfo      \
+perl-TermReadKey      \
+perl-Test-Fatal       \
+perl-Test-MockObject  \
+perl-Test-Simple      \
+perl-Test-Pod         \
+perl-Test-Requires    \ 
+perl-Try-Tiny         \
+perl-Unicode-String   \
+perl-URI              \
+perl-Proc-ProcessTable \
 cpanminus
 
-sudo cpanm Authen::NTLM Data::Uniqid IO::Tee JSON::WebToken JSON::WebToken::Crypt::RSA Test::Mock::Guard
+sudo cpanm Authen::NTLM Data::Uniqid IO::Tee \
+     JSON::WebToken JSON::WebToken::Crypt::RSA \
+     Test::Mock::Guard \
+     Encode::IMAPUTF7
 
 wget -N https://imapsync.lamiral.info/imapsync
 

INSTALL.d/INSTALL.ArchLinux.txt

@@ -1,5 +1,5 @@
 #!/bin/cat 
-# $Id: INSTALL.ArchLinux.txt,v 1.3 2018/09/03 02:00:22 gilles Exp gilles $
+# $Id: INSTALL.ArchLinux.txt,v 1.6 2022/05/17 14:31:44 gilles Exp gilles $
 
 This documentation is also located online at 
 https://imapsync.lamiral.info/INSTALL.d/
@@ -71,13 +71,15 @@ Commands to run:
    community/perl-test-nowarnings \
    community/perl-test-deep \
    extra/perl-try-tiny \
-   extra/perl-uri 
+   extra/perl-uri \
+   community/perl-file-tail
 
 Mandatory Perl modules via cpanm:
 
   cpanm Mail::IMAPClient \
    Unicode::String \
    Sys::MemInfo \
+   Encode::IMAPUTF7
 
 Other Perl modules, needed sometimes:
 
@@ -86,7 +88,8 @@ Other Perl modules, needed sometimes:
    JSON::WebToken::Crypt::RSA \
    Test::Mock::Guard \
    Test::Warn \
-   PAR::Packer
+   PAR::Packer \
+   Proc::ProcessTable
 
   
   

INSTALL.d/INSTALL.CPanel.txt

@@ -1,5 +1,5 @@
 #!/bin/cat 
-# $Id: INSTALL.CPanel.txt,v 1.6 2018/05/24 11:43:31 gilles Exp gilles $
+# $Id: INSTALL.CPanel.txt,v 1.7 2019/11/28 14:38:18 gilles Exp gilles $
 
 This documentation is also located online at 
 https://imapsync.lamiral.info/INSTALL.d/ 
@@ -26,5 +26,12 @@ A good test that shows also the basic example:
 
   imapsync
 
+
+If you want to update to the latest imapsync, then look at
+the Centos installation document called INSTALL.Centos.txt
+and also INSTALL.ANY.txt 
+
+You're on your own, I don't have access to a CPanel system.
+
 =======================================================================
 =======================================================================

INSTALL.d/INSTALL.Centos.txt

@@ -1,5 +1,5 @@
 #!/bin/cat 
-# $Id: INSTALL.Centos.txt,v 1.10 2019/02/05 18:20:42 gilles Exp gilles $
+# $Id: INSTALL.Centos.txt,v 1.32 2022/05/17 14:32:47 gilles Exp gilles $
 
 This documentation is also located online at 
 https://imapsync.lamiral.info/INSTALL.d/
@@ -9,7 +9,68 @@ https://imapsync.lamiral.info/INSTALL.d/INSTALL.Centos.txt
                Installing imapsync on CentOS
 =======================================================================
 
-There is one section for Centos 7 and one for Centos 6.
+There is one section for AlmaLinux and Centos 8,
+one for Centos 7,
+and one for Centos 6.
+
+
+=======================================================================
+               AlmaLinux and Centos 8
+=======================================================================
+
+How to install imapsync on AlmaLinux and Centos 8:
+
+  dnf install --enablerepo=powertools imapsync
+  imapsync --testslive
+
+
+For Centos 8 End Of Life fix:
+https://www.centos.org/centos-linux-eol/
+
+If you encounter the error 
+"Error: Failed to download metadata for repo 'appstream': Cannot prepare internal mirrorlist: No URLs in mirrorlist"
+
+Do this:
+
+  sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-Linux-*
+  sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-Linux-*
+  dnf update
+
+To upgrade to CentOS stream:
+
+  dnf install centos-release-stream
+  dnf swap centos-{linux,stream}-repos
+  dnf distro-sync
+
+Taken from 
+https://stackoverflow.com/questions/70926799/centos-through-vm-no-urls-in-mirrorlist
+
+and tested successfully on one of my hosts, an ex-Centos8/CentOS-stream now.
+
+=======================================================================
+               AlmaLinux and Centos 8 and latest imapsync
+=======================================================================
+
+At the time of this writing (December 2021), the powertools repository 
+contains imapsync release 1.977, which is not the latest available 
+imapsync release. (1.977 date is 2019/12/23 while 2.174 is 2021/12/14)
+
+In order to install the latest imapsync I suggest the following process:
+
+Install the powertools imapsync 1.977 release via dnf like mentioned above.
+It will install the needed packages, Perl itself and Perl modules 
+for release 1.977. 
+
+The imapsync 2.206 and next ones needs the Perl module:
+
+   dnf install perl-Proc-ProcessTable
+
+  wget -N https://imapsync.lamiral.info/imapsync
+  chmod +x imapsync
+  ./imapsync --testslive
+  ./imapsync --tests
+
+You may get an even more recent imapsync release this way.
 
 =======================================================================
                Centos 7
@@ -30,8 +91,8 @@ A good test that shows also the basic example:
 
 A live test:
 
-  imapsync --testslive
-  
+  imapsync --testslive --tls1 --ssl2
+
 Unit tests:
 
   imapsync --tests
@@ -40,8 +101,9 @@ Unit tests:
                Centos 7 and latest imapsync
 =======================================================================
 
-At the time of this writing (April 2018), the epel repository contains 
-imapsync release 1.727, which is not the latest available release.
+At the time of this writing (March 2020), the epel7 repository still
+contains imapsync release 1.727, which is not the latest available 
+imapsync release. (1.727 date is 2016/08/19 while 2.174 is 2021/12/14)
 
 In order to install the latest imapsync I suggest the following process:
 
@@ -65,18 +127,38 @@ Then:
  perl-Test-Fatal \
  perl-Test-Mock-Guard \
  perl-Test-Requires \
- perl-Test-Deep
+ perl-Test-Deep \
+ perl-File-Tail \
+ perl-Unicode-String \
+ perl-Test-NoWarnings \
+ perl-Test-Simple \
+ perl-Test-Warn \
+ perl-Sub-Uplevel \
+ perl-Proc-ProcessTable \
+ ca-certificates
 
-  wget https://imapsync.lamiral.info/imapsync
+Also, run the following commands:
 
+  cpanm Encode::IMAPUTF7 
+  wget -N https://imapsync.lamiral.info/imapsync
   chmod +x imapsync
   ./imapsync --testslive
 
-Now you have the latest imapsync on Centos 7.
+Now you should have the latest imapsync locally working on Centos 7.
+
+If you want to install it and replace the old one:
+
+  mv /usr/bin/imapsync  /usr/bin/imapsync_old
+  cp ./imapsync /usr/bin/imapsync
+
+  imapsync --version
+
+should give the latest release number instead of 1.727
+(It is 1.979 at the time of this writing)
 
 If you want to install an online imapsync service
-like /X then you also need a recent CGI.pm,
-add with cpanm:
+like /X then you also need a recent decent CGI.pm,
+add it with cpanm:
 
   cpanm CGI 
 
@@ -134,7 +216,10 @@ Then:
  perl-Test-Fatal \
  perl-Test-Mock-Guard \
  perl-Test-Requires \
- perl-Test-Deep
+ perl-Test-Deep \
+ perl-File-Tail \
+ perl-Unicode-String \
+ perl-Proc-ProcessTable 
 
   wget https://imapsync.lamiral.info/imapsync
 

INSTALL.d/INSTALL.Darwin.txt

@@ -1,31 +1,62 @@
 #!/bin/cat
-# $Id: INSTALL.Darwin.txt,v 1.19 2018/12/06 10:07:58 gilles Exp gilles $
+# $Id: INSTALL.Darwin.txt,v 1.39 2022/05/17 14:31:44 gilles Exp gilles $
 
 This documentation is also located online at 
 https://imapsync.lamiral.info/INSTALL.d/
 https://imapsync.lamiral.info/INSTALL.d/INSTALL.Darwin.txt
 
+
+Usual users should follow the A) section only, with the exception 
+of reading B) before if you use a Catalina system.
+
+A) Installing imapsync binary on Darwin / Mac OS X
+B) Installing imapsync on Catalina
+C) Installing imapsync script on Darwin / Mac OS X with brew
+D) Installing imapsync script on Darwin / Mac OS X the way I do
+E) Building imapsync binary on Darwin / Mac OS X
+
+
 =======================================================================
-               Installing imapsync binary on Darwin / Mac OS X
+A) Installing imapsync binary on Darwin / Mac OS X
 =======================================================================
 
 There is a standalone imapsync binary for Mac OS X 
 called "imapsync_bin_Darwin" (without the quotes),
-available in the compressed tarball called imapsync-1.xxx.tgz
-where 1.xxx is the version number. I suppose this tarball is put
-under your $HOME directory, let say /Users/gilles/, but you can
-put it anywhere.
+available in the compressed tarball called imapsync-2.200.tgz
+where 2.200 is the imapsync version number. 
 
-Open a terminal: /Applications/Utilities/Terminal double-click on Terminal.
+In case you haven't bought it yet or forgot where the download link is,
+check https://imapsync.lamiral.info/dist/
 
-Untar the tarball where you want:
+Download the tarball imapsync-2.200.tgz, not the binary imapsync_bin_Darwin
+because the binary is also in the tarball and there are other useful files
+in the tarball. The binary present in /dist/ is just there for a fast upgrade
+when you need one.
+
+Suppose this tarball imapsync-2.200.tgz is downloaded under your $HOME directory, 
+lets say /Users/gilles/, but you can put it anywhere. (But obviously your $HOME
+directory is not /Users/gilles/, it maybe /Users/john/ or /Users/zoey/ where John
+or Zoey is your login name.)
+
+Open a terminal: /Applications/Utilities/Terminal, double-click on Terminal.
+
+Untar the tarball:
 
   cd
-  tar xzvf  /Users/gilles/imapsync-1.xxx.tgz
+  pwd
+  tar xzvf imapsync-2.200.tgz
 
-Go into the directory imapsync-1.xxx
+In case the previous command fails, it means the tarball file called imapsync-2.200.tgz
+is not in your $HOME directory, you may have downloaded it elsewhere on the file system.
+A way to find it is the command:
 
-  cd imapsync-1.xxx
+  find / | grep imapsync 
+
+
+Now that the tarball is extracted, it created a directory called imapsync-2.200/ 
+Go into the directory imapsync-2.200 with the command:
+
+  cd imapsync-2.200
 
 
 First let's have a simple run to see if imapsync_bin_Darwin works. 
@@ -42,8 +73,8 @@ real IMAP server accounts:
 If this sync works fine then imapsync_bin_Darwin is ready for any 
 imap account synchronization. 
 
-When reading the documentation with imapsync command lines 
-examples, you have to replace the command "imapsync" by "imapsync_bin_Darwin"
+When reading the documentation with imapsync command lines examples, 
+you have to replace the command "imapsync" by "imapsync_bin_Darwin"
 For example, instead of the command:
 
   ./imapsync \
@@ -56,8 +87,66 @@ you have to use:
     --host1 test1.lamiral.info --user1 test1 --password1 secret1 \
     --host2 test2.lamiral.info --user2 test2 --password2 secret2
 
+
+The script examples/imapsync_example_darwin.sh is ready to use,
+it is a copy of examples/imapsync_example.sh adapted to Mac users.
+
+
+Be careful the way you edit the script files, use a text editor. 
+Do not use a word processor because word processors add 
+or use special formating characters that will break the shell
+scripts. If you use TextEdit, use the text mode.
+
+In the terminal, try:
+
+  sh examples/imapsync_example_darwin.sh
+
+or copy it and run your copy instead:
+
+  cp examples/imapsync_example_darwin.sh mysync.sh
+  sh mysync.sh
+
+Now read on the tutorial
+https://imapsync.lamiral.info/doc/TUTORIAL_Unix.html
+in order to complete your formation on imapsync.
+
+If you need to sync or migrate many accounts, 
+the script examples/sync_loop_darwin.sh is also ready to use.
+
+
 =======================================================================
-    Installing imapsync script on Darwin / Mac OS X with brew
+B) Installing imapsync on Catalina
+=======================================================================
+
+Caveat Catalina: The binary imapsync_bin_Darwin is detected as malware
+or similar on the latest Mac OS X named Catalina. 
+It's Catalina new security policy. Imapsync is not a malware or similar.
+All other Mac OS X releases are ok.
+
+In order to pass away this detection, follow these steps:
+
+Run imapsync_bin_Darwin
+After this first launch, the security asks to authorize the unknown 
+developer program. Set up the root of the imapsync_bin_Darwin file 
+and then run it again or any of the .sh files, it should run ok 
+without any notice any more.
+
+Sources of this tip:
+https://www.quora.com/What-does-Can-t-be-opened-because-it-s-integrity-cannot-be-verified-mean-on-OSX-Catalina
+https://github.com/fastlane/fastlane/issues/15186#issuecomment-532047545
+https://github.com/neovim/neovim/issues/11011#issuecomment-531369505
+
+Other way, on the binary: Control + Right Click -> Open
+
+See also the last section of https://support.apple.com/en-us/HT202491
+"How to open an app that hasn't been notarized or is from an unidentified developer"
+
+I don't own a Catalina computer so I can't experiment on this nor make
+imapsync directly ok with Catalina. You're richer than me :-)
+
+
+=======================================================================
+C) Installing imapsync script on Darwin / Mac OS X with brew
 =======================================================================
 
 In case you have brew installed on your system ( see https://brew.sh/ ),
@@ -68,31 +157,39 @@ install imapsync with the command:
 Do a real test with imap server test.lamiral.info:
 
   imapsync --testslive
-  
+
+If you encouter an error like "String.c: loadable library and perl binaries
+are mismatched (got handshake key 0xc500080, needed 0xc400080)",
+then install imapsync from source with the command:
+
+  brew install -s imapsync
+
+Thanks to Alessandro for this tip!
 
 =======================================================================
-    Installing imapsync script on Darwin / Mac OS X the way I do
+D) Installing imapsync script on Darwin / Mac OS X the way I do
 =======================================================================
 
 This part is only for advanced Unix users, or brave users.
 
 The "make" command is a prerequisite to build some Perl modules.
-Install make in case it is not already installed.
+Install the "make" command in case it is not already installed.
 
 First let us install cpanminus locally in ~/perl5
 
-  wget --no-check-certificate -O- http://cpanmin.us | perl - -l ~/perl5 App::cpanminus local::lib
+  curl -L https://cpanmin.us | perl - -l ~/perl5 App::cpanminus local::lib
 
 Then take this install into account in the current environment
 
-  eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib`
-  perl -I ~/perl5/lib/perl5 -Mlocal::lib
+  perl -I ~/perl5/lib/perl5 -Mlocal::lib        # just to see the variables
+  eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib` # doing the variables assignments
 
 If you want to have always this setting in your environment then run the commands
   
   echo 'eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib`' >> ~/.profile
   echo 'export MANPATH=$HOME/perl5/man:$MANPATH' >> ~/.profile
   cat ~/.profile 
+  . ~/.profile
 
 Now let's update the standard CPAN Perl module
 
@@ -105,18 +202,27 @@ helps to verify what is needed to install on your system
   sh prerequisites_imapsync
 
   cpanm Authen::NTLM
-  cpanm File::Copy::Recursive IO::Tee
-  cpanm Mail::IMAPClient
-  cpanm Readonly
+  cpanm IO::Tee 
+  cpanm Mail::IMAPClient # File::Copy::Recursive?
+  
   cpanm Unicode::String
+  cpanm Sys::MemInfo
+  cpanm File::Tail
+  cpanm Proc::ProcessTable
+
+  cpanm Test::MockObject
+
+  cpanm Readonly
   cpanm Data::Uniqid
   cpanm JSON::WebToken
 
 We're ready to install and test the latest imapsync
 
-  wget -c http://imapsync.lamiral.info/imapsync
+  curl -L http://imapsync.lamiral.info/imapsync > imapsync
+  chmod +x imapsync
   ./imapsync
-  ./imapsync --modules
+  ./imapsync --testslive 
+  ./imapsync --tests
 
 You can rerun "sh prerequisites_imapsync" 
 to verify everything is ok:
@@ -126,18 +232,14 @@ to verify everything is ok:
 When everything is ok the script execution ends with this sentence
 "All needed modules are already installed"
 
-Now imapsync should work on your system.
-
-  ./imapsync --testslive
-
 =======================================================================
-               Building imapsync binary on Darwin / Mac OS X
+E) Building imapsync binary on Darwin / Mac OS X
 =======================================================================
 
 cpanm Module::ScanDeps
 cpanm PAR::Packer
 
-pp -o imapsync.bin imapsync 
+pp -x -u -o imapsync.bin imapsync 
 
 ./imapsync.bin 
 ./imapsync.bin  --testslive
@@ -145,12 +247,5 @@ pp -o imapsync.bin imapsync
 ./imapsync.bin  --module
 
 
-=======================================================================
-               Installing imapsync Mac OS X Leopard
-=======================================================================
-
-Mac OS X Leopard has perl 5.8.8 and an old openssl version.
-
-
 =======================================================================
 =======================================================================

INSTALL.d/INSTALL.Debian.txt

@@ -1,47 +1,85 @@
 #!/bin/cat 
-# $Id: INSTALL.Debian.txt,v 1.25 2018/11/20 10:28:33 gilles Exp gilles $
+# $Id: INSTALL.Debian.txt,v 1.38 2022/05/17 14:29:36 gilles Exp gilles $
 
 This documentation is also located online at 
 https://imapsync.lamiral.info/INSTALL.d/
 https://imapsync.lamiral.info/INSTALL.d/INSTALL.Debian.txt
 
 
-There is three install sections in this document,
-one for Debian 9 Stretch
-one for Debian 8 Jessie
-one for Debian 7 Wheezy
+There are four install sections in this document,
+1) one for Debian 9 Stretch, Debian 10 Buster, Debian 11 Bullseye, Debian 12 Bookworm
+2) one for all the Debian, called "After installing the dependencies".
+3) one for Debian 8 Jessie (obsolete)
+4) one for Debian 7 Wheezy (even more obsolete)
 
-First a call to Debian packagers
+After reading and following 1) or 3) or 4), you'll be invited to read and follow 
+the section "2) After installing the dependencies"
+
+What is your Debian release?
+Type on a terminal:
+
+    cat /etc/debian_version
+
+12.x => Bookworm
+11.x => Bullseye
+10.x => Buster
+ 9.x => Stretch
+ 8.x => Jessie
+ 7.x => Wheezy
+
+where x is any number between 0 and 9, x is minor and to be forgotten.
+
+
+First, a call to Debian packagers
 Why imapsync is not in Debian as a package?
 Imapsync used to be in Debian from 2005-04-25 (release 1.125) to 2011-01-25 (release 1.315).
 It could be in Debian nowadays, as always, my last words on this discussion were 
-"Do what you want, I promise I won't complain anymore about the fact imapsync is on Debian or not."
+"Do what you want, I promise I won't complain anymore about the fact imapsync is in Debian or not."
 https://lists.debian.org/debian-legal/2011/01/msg00058.html
-The license is now "No limits to do anything with this work and this license".
-Feel free.
+The Imapsync license is "No limits to do anything with this work and this license".
+So feel free!
 See also a more detailed story at
 https://lists.debian.org/debian-user/2016/11/msg00849.html
 
 
 =======================================================================
-               Installing imapsync on Debian 9 Stretch 
+  1) Installing imapsync on Debian 9 Stretch, 10 Buster, 11 Bullseye
 =======================================================================
 
-Here is the command to install imapsync dependencies.
+How to install imapsync dependencies on a Debian system?
+
 The repositories are the classical ones, ie, a
 /etc/apt/sources.list file with:
 
+# Debian 12 Bookworm
+deb http://deb.debian.org/debian bookworm main contrib
+
+or
+
+# Debian 11 Bullseye
+deb http://deb.debian.org/debian bullseye main contrib
+
+or
+
+# Debian 10 Buster
+deb http://deb.debian.org/debian buster main contrib
+
+or
+
+# Debian 9 Stretch
 deb http://deb.debian.org/debian stretch main contrib
 
+
 You need root privilege to run it, be root or use sudo.
 
 apt install -y            \
-  libjson-webtoken-perl   \
   libauthen-ntlm-perl     \
   libcgi-pm-perl          \
   libcrypt-openssl-rsa-perl   \
   libdata-uniqid-perl         \
+  libencode-imaputf7-perl     \
   libfile-copy-recursive-perl \
+  libfile-tail-perl        \
   libio-socket-inet6-perl  \
   libio-socket-ssl-perl    \
   libio-tee-perl           \
@@ -49,6 +87,7 @@ apt install -y            \
   libjson-webtoken-perl    \
   libmail-imapclient-perl  \
   libparse-recdescent-perl \
+  libproc-processtable-perl \
   libmodule-scandeps-perl  \
   libreadonly-perl         \
   libregexp-common-perl    \
@@ -63,19 +102,64 @@ apt install -y            \
   libtest-deep-perl        \
   libtest-warn-perl        \
   make                     \
+  time                     \
   cpanminus
 
 
 After installing the dependencies, imapsync should be working,
-go to the section "After installing the dependencies" below.
+go to the section "2) After installing the dependencies" below.
+
+=======================================================================
+  2) After installing the dependencies
+=======================================================================
+
+You don't have to be root to test and use imapsync.
+
+Take imapsync either on github:
+
+  wget -N https://raw.githubusercontent.com/imapsync/imapsync/master/imapsync
+
+or be on the cutting edge with the upstream site:
+
+  wget -N https://imapsync.lamiral.info/imapsync
+
+Add execution permission to the downloaded script:
+
+  chmod +x imapsync
+  
+Check the dependencies and print also the basic example:
+
+  ./imapsync
+
+Perform a live test showing imapsync job:
+
+  ./imapsync --testslive
+
+Now install imapsync on the system (need root privileges again):
+
+  cp imapsync /usr/bin/
+
+That's finished for the installation part.
+You can now use imapsync without knowing where it is located
+on the system:
+
+  imapsync 
+
+Now go read the menu http://imapsync.lamiral.info/#doc
+and start with the tutorial 
+https://imapsync.lamiral.info/doc/TUTORIAL_Unix.html
+
+Feedback is welcome!
+
 
 
 =======================================================================
-             Installing imapsync on Debian 8 Jessie 
+ 3) Installing imapsync on Debian 8 Jessie 
 =======================================================================
 
-Here are the commands to install imapsync dependencies.
-You need root priviledge to run them.
+How to install imapsync dependencies on a Debian system?
+
+You need root privilege.
 
 The first command installs standard Debian packages:
 apt install           \
@@ -83,7 +167,9 @@ apt install           \
   libcgi-pm-perl      \
   libcrypt-openssl-rsa-perl   \
   libdata-uniqid-perl         \
+  libencode-imaputf7-perl     \
   libfile-copy-recursive-perl \
+  libfile-tail-perl        \
   libio-socket-inet6-perl  \
   libio-socket-ssl-perl    \
   libio-tee-perl           \
@@ -102,6 +188,7 @@ apt install           \
   libtest-nowarnings-perl  \
   libtest-deep-perl        \
   libtest-warn-perl        \
+  gcc                      \
   make                     \
   cpanminus
 
@@ -136,45 +223,49 @@ the module JSON::WebToken with the command:
   cpanm JSON::WebToken
 
 After installing the dependencies, imapsync should be working,
-go to the section "After installing the dependencies" below.
+go to the section "2) After installing the dependencies" above.
 
 
 =======================================================================
-               Installing imapsync on Debian 7 Wheezy 
+  4) Installing imapsync on Debian 7 Wheezy 
 =======================================================================
 
-Here are the two commands to install imapsync dependencies.
-You need root priviledge to run them.
+How to install imapsync dependencies on a Debian system?
+
+You need root privilege to run the commands.
 
 The first command installs standard Debian packages:
 
-  apt-get install    \
-libauthen-ntlm-perl  \
-libclass-load-perl   \
-libcrypt-openssl-rsa-perl \
-libdigest-hmac-perl  \
-libfile-copy-recursive-perl \
-libio-compress-perl     \
-libio-socket-inet6-perl \
-libio-socket-ssl-perl   \
-libio-tee-perl          \
-libmail-imapclient-perl \
-libmodule-implementation-perl \
-libmodule-runtime-perl  \
-libmodule-scandeps-perl \
-libpackage-stash-perl   \
-libpackage-stash-xs-perl \
-libpar-packer-perl      \
-libreadonly-perl        \
-libterm-readkey-perl    \
-libtest-fatal-perl      \
-libtest-pod-perl        \
-libtest-requires-perl   \
-libtest-simple-perl     \
-libunicode-string-perl  \
-liburi-perl             \
-make                    \
-cpanminus
+apt-get install    \
+  libauthen-ntlm-perl  \
+  libclass-load-perl   \
+  libcrypt-openssl-rsa-perl \
+  libdigest-hmac-perl  \
+  libencode-imaputf7-perl     \
+  libfile-copy-recursive-perl \
+  libfile-tail-perl        \
+  libio-compress-perl     \
+  libio-socket-inet6-perl \
+  libio-socket-ssl-perl   \
+  libio-tee-perl          \
+  libmail-imapclient-perl \
+  libmodule-implementation-perl \
+  libmodule-runtime-perl  \
+  libmodule-scandeps-perl \
+  libpackage-stash-perl   \
+  libpackage-stash-xs-perl \
+  libpar-packer-perl      \
+  libreadonly-perl        \
+  libterm-readkey-perl    \
+  libtest-fatal-perl      \
+  libtest-pod-perl        \
+  libtest-requires-perl   \
+  libtest-simple-perl     \
+  libunicode-string-perl  \
+  liburi-perl             \
+  gcc                     \
+  make                    \
+  cpanminus
 
 
 The following second command installs "manually" the Perl modules 
@@ -200,40 +291,7 @@ the module JSON::WebToken with the command:
   cpanm JSON::WebToken
 
 
-After installing the dependencies, imapsync should be working.
+After installing the dependencies, imapsync should be working,
+then go to the section "2) After installing the dependencies" above.
 
-=======================================================================
-                 After installing the dependencies
-=======================================================================
-
-You don't have to be root to test and use imapsync.
-
-Take imapsync either on github or at the upstream site.
-
-Add execution permission to the downloaded script:
-
-  chmod +x imapsync
-  
-Check the dependencies and print also the basic example:
-
-  ./imapsync
-
-Perform a live test showing imapsync job:
-
-  ./imapsync --testslive
-
-Now install imapsync on the system (need root priviledges again):
-
-  cp imapsync /usr/bin/
-
-That's finished for the installation part.
-You can now use imapsync without knowing where it is located
-on the system:
-
-  imapsync 
-
-Now go to read http://imapsync.lamiral.info/#doc
-start with the tutorial.
-
-=======================================================================
 =======================================================================

INSTALL.d/INSTALL.Docker.txt

@@ -1,34 +0,0 @@
-#!/bin/cat
-# $Id: INSTALL.Docker.txt,v 1.4 2018/09/13 17:50:06 gilles Exp gilles $
-
-This documentation is also located online at 
-https://imapsync.lamiral.info/INSTALL.d/
-https://imapsync.lamiral.info/INSTALL.d/INSTALL.Docker.txt
-
-=======================================================================
-               Installing imapsync docker image
-=======================================================================
-
-==========================
-== Installation & usage ==
-==========================
-
-That's all you have to run:
-
-  docker run gilleslamiral/imapsync imapsync <usual imapsync arguments>
-
-=========================
-== Installation alone  ==
-==========================
-
-To install or update the imapsync image, run:
-
-  docker pull gilleslamiral/imapsync
-
-== Docker hub location ==
-
-https://hub.docker.com/r/gilleslamiral/imapsync/
-
-
-=======================================================================
-=======================================================================

INSTALL.d/INSTALL.Docker_build.txt

@@ -1,5 +1,5 @@
 #!/bin/cat 
-# $Id: INSTALL.Docker_build.txt,v 1.8 2018/05/24 11:43:31 gilles Exp gilles $
+# $Id: INSTALL.Docker_build.txt,v 1.17 2022/05/17 14:29:36 gilles Exp gilles $
 
 This documentation is also located online at 
 https://imapsync.lamiral.info/INSTALL.d/
@@ -9,22 +9,68 @@ https://imapsync.lamiral.info/INSTALL.d/INSTALL.Docker_build.txt
                Building an imapsync docker image from Debian
 =======================================================================
 
+Nearly verbatim copy from https://imapsync.lamiral.info/INSTALL.d/Dockerfile 
+(verbatim but the Dockerfile rcs tag $ removed because it would conflict)
 
-# Dockerfile for building a docker imapsync image
+## Dockerfile for building a docker imapsync image
 
-FROM debian:stretch
+# Id: Dockerfile,v 1.42 2022/04/04 19:16:50 gilles Exp gilles $
+# I use the following command to build the image:
+#
+#  docker build -t gilleslamiral/imapsync . 
+#
+# where this Dockerfile is in the current directory
+# 
+# This Dockerfile builds an image with two imapsync
+# 1) One comes from https://imapsync.lamiral.info/imapsync
+#    and goes to /usr/bin/imapsync in the Docker image
+#    It is used with the command:
+#
+#    docker run gilleslamiral/imapsync imapsync ...
+# 
+# 2) One comes from the local file ./imapsync
+#    and goes to /imapsync in the Docker image
+#    It is used with the command:
+#
+#    docker run gilleslamiral/imapsync /imapsync ...
 
-LABEL maintainer "gilles@lamiral.info"
 
-RUN apt-get update \
+# I thank you very much
+# I like thanks
+# I like stars
+# I like money to keep on doing this stuff
+
+# Number of imapsync images pulled so far (2022_04_04): 673551
+# Command used:
+# curl -s https://hub.docker.com/v2/repositories/gilleslamiral/imapsync/ | jq '.pull_count'
+
+# Debian Bullseye is Debian 11
+
+FROM debian:bullseye 
+
+LABEL   maintainer="Gilles LAMIRAL <gilles@lamiral.info>" \
+        description="Imapsync" \
+        documentation="https://imapsync.lamiral.info/#doc"
+
+# I put a copy of the Dockerfile in the image itself
+# It can help maintenance, can't it?
+# Also put optionally my local and usually more recent imapsync on /, for testing purpose
+
+COPY Dockerfile imapsyn[c] prerequisites_imapsyn[c] /
+
+RUN set -xe && \
+  apt-get update \
   && apt-get install -y \
-  libjson-webtoken-perl \
   libauthen-ntlm-perl \
   libcgi-pm-perl \
   libcrypt-openssl-rsa-perl \
   libdata-uniqid-perl \
+  libencode-imaputf7-perl \
   libfile-copy-recursive-perl \
+  libfile-tail-perl \
+  libio-compress-perl \
   libio-socket-ssl-perl \
+  libio-socket-inet6-perl \
   libio-tee-perl \
   libhtml-parser-perl \
   libjson-webtoken-perl \
@@ -32,6 +78,7 @@ RUN apt-get update \
   libparse-recdescent-perl \
   libmodule-scandeps-perl \
   libpar-packer-perl \
+  libproc-processtable-perl \
   libreadonly-perl \
   libregexp-common-perl \
   libsys-meminfo-perl \
@@ -43,17 +90,39 @@ RUN apt-get update \
   libwww-perl \
   procps \
   wget \
-  && rm -rf /var/lib/apt/lists/*
+  make \
+  cpanminus \
+  lsof \
+  ncat \
+  openssl \
+  ca-certificates \
+  && rm -rf /var/lib/apt/lists/* \
+  && cpanm IO::Socket::SSL
 
-RUN wget -N https://imapsync.lamiral.info/imapsync \
-  && cp imapsync /usr/bin/imapsync \
-  && chmod +x /usr/bin/imapsync
+# I added the IO::Socket::SSL update to avoid the annoying, confusing and useless warning
+# DEBUG: .../IO/Socket/SSL.pm:1177: global error: Undefined SSL object
 
-USER nobody
+RUN set -xe \
+  && cd /usr/bin/ \
+  && pwd \
+  && wget -N --no-check-certificate https://imapsync.lamiral.info/imapsync \
+        https://imapsync.lamiral.info/prerequisites_imapsync \
+        https://raw.githubusercontent.com/google/gmail-oauth2-tools/master/python/oauth2.py \
+  && chmod +x imapsync oauth2.py \
+  && /usr/bin/imapsync --testslive && /usr/bin/imapsync --tests # just_a_comment_to_force_update 2022_04_04_21_16_50
+
+USER nobody:nogroup
+
+ENV HOME /var/tmp/
+
+WORKDIR /var/tmp/
+
+STOPSIGNAL SIGINT
 
 CMD ["/usr/bin/imapsync"]
 
-# End of Dockerfile
+# 
+# End of imapsync Dockerfile 
 
 =======================================================================
 =======================================================================

INSTALL.d/INSTALL.FreeBSD.txt

@@ -1,5 +1,5 @@
 #!/bin/cat 
-# $Id: INSTALL.FreeBSD.txt,v 1.10 2018/07/03 04:29:11 gilles Exp gilles $
+# $Id: INSTALL.FreeBSD.txt,v 1.12 2022/05/17 14:31:44 gilles Exp gilles $
 
 This documentation is also located online at 
 https://imapsync.lamiral.info/INSTALL.d/
@@ -9,7 +9,7 @@ https://imapsync.lamiral.info/INSTALL.d/INSTALL.FreeBSD.txt
                Installing imapsync on FreeBSD
 =======================================================================
 
-Thanks to Kurt Jaeger pi@FreeBSD.org a imapsync package is available in FreeBSD  
+Thanks to Kurt Jaeger pi@FreeBSD.org, an imapsync package is available in FreeBSD  
 http://portsmon.freebsd.org/portoverview.py?category=mail&portname=imapsync
 So, to install imapsync you just have to type one command in a terminal:
 
@@ -30,6 +30,10 @@ install also:
 p5-Regexp-Common
 p5-Test-MockObject
 p5-CGI
+p5-Encode-IMAPUTF7
 
-Call to FreeBSD developer: It's also the new dependencies to add 
-for next the imapsync  package 1.882 
+
+Call to FreeBSD developer: These are also the new Perl module dependencies to add 
+for imapsync 2.206 and next ones:
+
+Proc::ProcessTable

INSTALL.d/INSTALL.OnlineUI.txt

@@ -1,5 +1,5 @@
 #!/bin/cat 
-# $Id: INSTALL.OnlineUI.txt,v 1.29 2019/02/18 09:24:18 gilles Exp gilles $
+# $Id: INSTALL.OnlineUI.txt,v 1.56 2022/05/19 08:55:50 gilles Exp gilles $
 
 This documentation is also located online at 
 https://imapsync.lamiral.info/INSTALL.d/
@@ -9,193 +9,300 @@ https://imapsync.lamiral.info/INSTALL.d/INSTALL.OnlineUI.txt
                Installing imapsync online
 =======================================================================
 
-Please consider this as relatively new and experimental.
-However, to a certain extent, I'm now confident with /X 
-since the /X service is up and running quite well since January 2017.
+
+I'm now confident with /X since the /X service is up and running quite
+well since January 2017. Anyway, if you run this service on your own,
+online, you take responsibility for it.
 
 =======================================================================
                     Hardware consideration
-                    
 
 RAM used per imapsync process, mean value: 230 MB.
 Average_bandwidth_rate: 345 KiB/s ~ 2.8 Mbps.
 Load mean: 0.8 on a CPU 4 cores "Intel(R) i5-2320 3.00GHz K8-class"
 
 
-
 =======================================================================
                     Installation
-                    
-You have to be a little familiar with what a CGI script is
-and how to activate a CGI script on the Apache 
-HTTP server, or any other HTTP server. Linux is also a 
-preferred platform. 
 
-I haven't tested yet this visual interface on Windows nor Mac,
-feedback is welcome from successes or failures on those platforms.
-Like imapsync is written, it should fail on Windows because of
-some hard coded Unix paths.
+You have to be a little familiar with what a CGI script is and how to
+activate a CGI script on the Apache HTTP server, or any other HTTP
+server. I have received demands to run it on the Ngnix HTTP server but
+I haven't played with it yet.  Linux is also a preferred platform (I
+run /X service on Linux and FreeBSD).
 
-The web visual user interface frontend /X is compounded in four
-files: a html5 file, a css file, a javascript file and
-a logo image:
+I have tested this visual interface on Mac. It works.  For now, it
+demands some skills few Mac users have.  Drop me a note in case you
+want to do that.
 
-* https://i005.lamiral.info/X/imapsync_form.html
-* https://i005.lamiral.info/X/imapsync_form.css
-* https://i005.lamiral.info/X/imapsync_form.js
-* https://i005.lamiral.info/X/logo_imapsync_Xn.png
+I have tested this visual interface on Windows, it fails on Windows
+because of some hardcoded Unix paths.  I'm working on it to be Windows
+ok but it's not done yet (May 2020).
+
+Some users have successfully installed a /X visual interface on
+Windows using a Linux VM machine.
+
+The web visual user interface frontend /X is compounded in four files:
+a html5 file, a CSS file, a javascript file, and a logo image:
+
+* https://imapsync.lamiral.info/X/imapsync_form_extra.html
+* https://imapsync.lamiral.info/X/imapsync_form.css
+* https://imapsync.lamiral.info/X/imapsync_form.js
+* https://imapsync.lamiral.info/X/logo_imapsync_Xn.png
 
 
-You can do a "view source" to see the html file as 
-it is written, and a "save" to get it locally.
-The three other files can be saved the same way or
-with wget.
+You can do a "view source" to see the HTML file as it is written, and
+a "save" to get it locally.  The three other files can be saved the
+same way or with a command named "wget". I strongly suggest using
+wget, see below the ready-to-use command lines.
 
-Those four files can be put anywhere on a web server,
-as long as they stand in the same directory. If you
-want to put them in different directories, just change
-the content of imapsync_form.html to reflect the change,
-ie, change the two lines referencing imapsync_form.css and imapsync_form.js 
-  href="imapsync_form.css" (near the beginning of imapsync_form.html)
-  src="imapsync_form.js"   (near the end of imapsync_form.html)
-I let you changing the image logo as an exercise.
+Those four files can be put anywhere on a web server, as long as they
+stand in the same directory. If you want to put them in different
+directories, just change the content of imapsync_form_extra.html to
+reflect the change, ie, change the two lines referencing
+imapsync_form.css and imapsync_form.js href="imapsync_form.css" (near
+the beginning of imapsync_form_extra.html) src="imapsync_form.js"
+(near the end of imapsync_form_extra.html) I let you change the image
+logo as an exercise, it's safe if you fail.
 
-The actual imap syncing work is done by imapsync acting as a CGI,
-the visual interface is only there to give imapsync the parameters
-needed for the sync.
+The actual imap syncing work is done by imapsync acting as a CGI, the
+visual interface is only there to give imapsync the parameters needed
+for the sync.
 
-Use at least Perl module CGI.pm release 4.08 (2014-10-18)
-to avoid the bug "Undefined subroutine CGI::multi_param".
-You can use the command cpanm to upgrade CGI.pm to its last version,
-it's the easiest way.
+Use at least Perl module CGI.pm release 4.08 (2014-10-18) to avoid the
+bug "Undefined subroutine CGI::multi_param".  You can use the command
+named cpanm to upgrade CGI.pm to its last version, it's the easiest
+way.
 
 Print the CGI.pm release with:
 
   perl -MCGI -e 'print "$CGI::VERSION\n"'
 
+If it is under release 4.08 (2014-10-18) then upgrade it with
+
+  cpanm CGI
+
+It is a good thing to remove the old one if it was installed by a
+distribution package, I let you this part as an exercise too. Ok, here
+is a way:
+
+  apt remove libcgi-pm-perl # for the Debian family
+  dnf remove perl-CGI       # for the Centos family
+
 To check and fix the Perl modules dependencies, run:
 
   cd
   wget -N https://imapsync.lamiral.info/prerequisites_imapsync
   sh prerequisites_imapsync
 
-In order to make imapsync work as a cgi, there is two conditions.
-First, imapsync has to work by itself on the web host.
-If imapsync doesn't work by itself, as a command line, 
-then it won't work as a cgi.
-Second, imapsync has to work by itself on the web host using
-the Unix user running the web server.
-Detailed examples to verify that will be provided further 
-in this document.
+To make imapsync work as a CGI script, there are three conditions.
 
-The imapsync_form.html file in action calls the CGI location
+First, imapsync has to work by itself on the web host.  If imapsync
+doesn't work by itself, as a command line, then it won't work as a CGI
+script.
+
+Second, imapsync has to work by itself on the web host using the Unix
+user running the webserver.
+
+Third, the file imapsync has to be considered as a cgi script.
+
+Command lines to provide and verify those three conditions will be
+provided further in this document, for the Debian family systems and
+for the Centos family systems. You are strongly advised to follow this
+commands if need and want help from me because I will first ask you to
+run them before searching what you did wrong.
+
+The imapsync_form_extra.html file in action calls the CGI location
 /cgi-bin/imapsync
 which has to be imapsync itself, the file script (not the directory).
 
-The very latest and relatively stable imapsync is at
+The very latest and relatively stable imapsync is
 https://imapsync.lamiral.info/imapsync
-It is the program file used verbatim for the service given at
-https://i005.lamiral.info/X/
+This file is the program file used verbatim for the service given at
+https://imapsync.lamiral.info/X/
 
-Copy the three files imapsync_form.* on a directory that is exported 
+Copy the three files imapsync_form.* on a directory that is exported
 by your HTTP server.
+
 Copy the imapsync script on the cgi-bin/ directory
-allowing CGIs and you'll have your own imapsync visual interface
-and service. The cgi-bin/ directory is usually outside the 
+allowing CGIs and you'll have your imapsync visual interface
+and service. The cgi-bin/ directory is usually outside the
 hierarchy exported to anybody by the HTTP server.
 
+The default Apache 2.4 timeout is 60 seconds, one minute, and 300
+secondes for older Apache, 5 minutes.  See
+https://httpd.apache.org/docs/2.4/mod/core.html#timeout
+
+I use "Timeout 3600", 3600 seconds, an hour. I chose this huge timeout
+value because imapsync can spend a long time without talking while
+getting the headers of huge folders of 100k messages.  If you intend
+to offer this service for huge mailboxes or for a long time, I
+strongly recommend you to set this "Timeout 3600" in the Apache
+configuration right now because you will sure end up with this timeout
+issue in a few months. You can search for timeouts in the Apache error
+log to see if you have timeout issues.
+
+Now that I have explained the general context for any system, I'll
+describe concrete examples on several systems, Debian/Ubuntu and
+Centos. Feedbacks show that the Centos process is easier in case you
+don't know very much any Linux distribution. But I have to add that if
+you don't know very much the Linux distribution you use, then you
+shouldn't install this imapsync service at all.
 
 
-Concrete example on a Debian server with Apache:
+=============================================================================
+A) Concrete example on a Debian server with Apache:
 
-Imapsync script place on the server disk:
+First, install Apache on your Debian system:
+
+  apt install apache2
+
+Imapsync script place on the server disk will be
 /usr/lib/cgi-bin/imapsync
 
-Get and install latest imapsync:
-
-  cd
-  wget -N -O /usr/lib/cgi-bin/imapsync  https://imapsync.lamiral.info/imapsync
-  chmod +x /usr/lib/cgi-bin/imapsync
-
- Check it like this:
-
-  cd
-  /usr/lib/cgi-bin/imapsync --testslive
-
-Assuming that the Unix account running Apache is www-data, 
-check that it will work under Apache with this command:
-
-  su -s /bin/sh -c 'SERVER_SOFTWARE=foo /usr/lib/cgi-bin/imapsync' www-data
-
-  wget -nv -S -O-  http://localhost/cgi-bin/imapsync?testslive=1
-
-It should print something like:
-Status: 200 OK to sync IMAP boxes. Load on bar is ...
-...
-
-
-This classical /cgi-bin directory is already configured 
+This classical /cgi-bin directory is usually already configured 
 in the Apache configuration file
 /etc/apache2/sites-available/default-ssl
 or
 /etc/apache2/sites-available/default
 
-The configuration file contains the following section
+This configuration file contains the following section
 somewhere, maybe in comments for now, ie, with 
-some # characters 
-at the beginning to make them ignored:
+some # characters at the beginning to make them ignored.
+If you don't find the following section, keep reading,
+the solution is below.
 
 ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
 <Directory "/usr/lib/cgi-bin">
     AllowOverride None
     # Next line "no-gzip 1" is to avoid output buffering, 
-    # clients can then see the log along the sync
+    # clients can then see the log during the sync
     SetEnv no-gzip 1
     Options +ExecCGI -MultiViews
-    Order allow,deny
-    Allow from all
+    
+    # Choose either one or the other, depending on your Apache version
+    # Lines beginning with # are ignored
+    
+    # For Apache 2.2
+    #Order allow, deny
+    #Allow from all
+    
+    # Apache 2.4
+    Require all granted
 </Directory>
 
-The UI front-end file place on the server disk in
-this example is
-/var/www/html/X/imapsync_form.html
-but it can be placed it anywhere on disk, the important
-thing is that it has to be served by the web server. 
+In recent Debian distributions you can activate this cgi 
+stuff with the following commands:
+
+  a2enmod cgi
+  a2enconf  serve-cgi-bin
+  /etc/init.d/apache2 restart
+
+If the cgi mode and the cgi-bin configuration are not activated then
+you may encounter a 404 error when, later, you will run the command
+wget -nv -S -O-  http://localhost/cgi-bin/imapsync?testslive=1
+
+That's all for the Apache Debian family configuration side.
+
+Now get, test, and install the latest imapsync:
+
+  cd
+  wget -N https://imapsync.lamiral.info/imapsync
+  chmod +x imapsync
+
+  # some basic tests
+  ./imapsync 
+  ./imapsync --testslive
+
+  cp imapsync /usr/lib/cgi-bin/
+
+Assuming that the Unix account running Apache is www-data, check that
+it will work under Apache with this command:
+
+  su -s /bin/sh -c 'SERVER_SOFTWARE=foo /usr/lib/cgi-bin/imapsync' www-data
+
+You should end with something like:
+Exiting with return value 0 (EX_OK: successful termination)
+
+Test that imapsync is considered a cgi with: 
+
+  wget -nv -S -O-  http://localhost/cgi-bin/imapsync?testslive=1
+
+The last command should print something like:
+Status: 200 OK to sync IMAP boxes. Load on bar is ...
+...
+Exiting with return value 0 (EX_OK: successful termination)
+
+If you get a 404 or a 5xx here then review the cgi installation and
+configuration part.
+
+
+You can also verify that the webserver is not buffering its output with the 
+command:
+
+  wget -nv -S -O-  'http://localhost/cgi-bin/imapsync?testslive=1&simulong=10'
+
+You should get the output as time goes on. If you don't get the output
+as time goes on, ie you see no output then all output at once, it
+means the webserver is buffering. Fix it with the "SetEnv no-gzip 1"
+described above.
+
+
+The UI front-end file place on the server disk in this example is
+/var/www/html/X/imapsync_form_extra.html
+but it can be placed anywhere on the disk, the important thing is that
+it has to be served by the webserver.
 
   mkdir /var/www/html/X/
   cd /var/www/html/X/
   wget -N \
-  https://i005.lamiral.info/X/imapsync_form.html \
-  https://i005.lamiral.info/X/imapsync_form.css \
-  https://i005.lamiral.info/X/imapsync_form.js \
-  https://i005.lamiral.info/X/logo_imapsync_Xn.png
-  ln -s imapsync_form.html index.html
+  https://imapsync.lamiral.info/X/imapsync_form_extra.html \
+  https://imapsync.lamiral.info/X/imapsync_form.css \
+  https://imapsync.lamiral.info/X/imapsync_form.js \
+  https://imapsync.lamiral.info/X/logo_imapsync_Xn.png
+  ln -s imapsync_form_extra.html index.html
 
 The imapsync process working directory in cgi mode is
 /var/tmp/imapsync_cgi/
-it is not configurable unless changing it in
-imapsync directly, it is hard-coded in imapsync. 
-In this directory will go the log files and
+it is not configurable unless changing it in imapsync directly, it is
+hard-coded in imapsync.  In this directory will go the log files and
 the pid files.
 
 Check 
-   http://yourhost/X/imapsync_form.html
+   http://yourhost/X/imapsync_form_extra.html
 or the safer
-  https://yourhost/X/imapsync_form.html
+  https://yourhost/X/imapsync_form_extra.html
 
-That's all for installing a /X service on Debian.
+Let's encrypt your site because credentials should never travel in
+clear form. Go to https://certbot.eff.org/instructions
 
+See the Troubleshooting section to fix the systemd Apache 
+PrivateTmp=true issue.
 
-Example on a Centos 7 server with httpd Apache:
+That's all for installing a /X service on a Debian family system.
+
+=============================================================================
+B) Here is a concrete example on a Centos 7 server with the Apache
+   webserver httpd:
+
+First, follow and apply the section "Centos 7 and latest imapsync" at
+https://imapsync.lamiral.info/INSTALL.d/INSTALL.Centos.txt
+
+Then:
+
+  yum install httpd
+  systemctl restart httpd
+
+  cpanm CGI
 
   mkdir /var/www/html/X/
   cd /var/www/html/X/
   wget -N \
-  https://i005.lamiral.info/X/imapsync_form.html \
-  https://i005.lamiral.info/X/imapsync_form.css \
-  https://i005.lamiral.info/X/imapsync_form.js \
-  https://i005.lamiral.info/X/logo_imapsync_Xn.png
-  ln -s imapsync_form.html index.html
+  https://imapsync.lamiral.info/X/imapsync_form_extra.html \
+  https://imapsync.lamiral.info/X/imapsync_form.css \
+  https://imapsync.lamiral.info/X/imapsync_form.js \
+  https://imapsync.lamiral.info/X/logo_imapsync_Xn.png
+  ln -s imapsync_form_extra.html index.html
   
   cd
   wget -N https://imapsync.lamiral.info/imapsync
@@ -205,17 +312,241 @@ Example on a Centos 7 server with httpd Apache:
   ./imapsync --testslive
 
   cp imapsync /var/www/cgi-bin/
+
+Assuming that the Unix account running Apache is "apache",
+which is the default Apache user on Centos system,
+check that it will work under Apache with this command:
+
+# a real synchronization but not in cgi context
+  cd /tmp
+  su -s /bin/sh -c '/var/www/cgi-bin/imapsync --testslive' apache
+  
+# in cgi context but just the imapsync command with no parameter
+  cd 
   su -s /bin/sh -c 'SERVER_SOFTWARE=foo /var/www/cgi-bin/imapsync' apache
+  
+# a real synchronization in cgi context
   wget -nv -S -O-  http://localhost/cgi-bin/imapsync?testslive=1
 
+The last command should print something like:
+Status: 200 OK to sync IMAP boxes. Load on bar is ...
+...
 
-Check 
-   http://yourhost/X/imapsync_form.html
+You can also verify that the webserver is not buffering its output
+with the command:
+
+  wget -nv -S -O-  'http://localhost/cgi-bin/imapsync?testslive=1&simulong=10'
+
+You should get the output as time goes on. If you don't, no output
+then all output at once, it means the webserver is buffering. Fix it
+with the "SetEnv no-gzip 1" described above.
+
+Now check 
+   http://yourhost/X/imapsync_form_extra.html
 or the safer
-  https://yourhost/X/imapsync_form.html
+  https://yourhost/X/imapsync_form_extra.html
 
 That's all for installing a /X service on Centos 7.
 
+
+B bis) How about Centos 8?
+
+Follow the procedure for Centos 7. While imapsync is ok on the command
+line, you will encounter some permission denied in the CGI
+context. Something like:
+
+wget -nv -S -O-  http://localhost/cgi-bin/imapsync?testslive=1
+...
+Host1 failure: can not open imap connection on host1
+[test1.lamiral.info] with user [test1]: Unable to connect to
+test1.lamiral.info: Permission denied
+
+The issue might come from SELinux. I haven't dig into SELinux enough
+to give you the commands that will allow imapsync online and only it
+while maintaining SELinux in enforcing mode.
+
+Quick solution:
+
+   getsebool    httpd_can_network_connect    # should show --> off
+   setsebool -P httpd_can_network_connect=1
+   getsebool    httpd_can_network_connect    # should show --> on
+   wget -nv -S -O-  http://localhost/cgi-bin/imapsync?testslive=1  # no more "Permission denied"
+
+The -P option installs the rule permanently, even after a reboot
+
+To go back to the previous state:
+
+   getsebool    httpd_can_network_connect    # should show --> on
+   setsebool -P httpd_can_network_connect=0
+   getsebool    httpd_can_network_connect    # should show --> off
+   
+
+Nota bene
+=========
+
+You may also want to avoid being placed by systemd in a directory like
+(where xxx are crypto hash characters):
+
+/var/tmp/systemd-private-xxx-httpd.service-xxx/tmp/
+
+In that case, see the Troubleshooting section below.
+
+
+=======================================================================
+===================   Bandwidth statistics   ==========================
+=======================================================================
+
+If you want the bandwidth statistics like the ones at the bottom of
+the page and following the image link, more detailed at
+https://imapsync.lamiral.info/vnstat/vnstati.html
+
+Those stats are generated by vnstat
+https://humdi.net/vnstat/
+
+Vnstat is already available as a package in most Linux distros.
+
+The images are generated by the following commands, every minute:
+vnstati -s  -o /var/www/html/vnstat/vnstat_s.png
+vnstati -h  -o /var/www/html/vnstat/vnstat_h.png
+vnstati -hg -o /var/www/html/vnstat/vnstat_hg.png
+vnstati -hs -o /var/www/html/vnstat/vnstat_hs.png
+vnstati -d  -o /var/www/html/vnstat/vnstat_d.png
+vnstati -m  -o /var/www/html/vnstat/vnstat_m.png
+vnstati -y  -o /var/www/html/vnstat/vnstat_y.png
+vnstati -t  -o /var/www/html/vnstat/vnstat_t.png
+vnstati -vs -o /var/www/html/vnstat/vnstat_vs.png
+vnstati -5  -o /var/www/html/vnstat/vnstat_5.png
+
+
+
+
+=======================================================================
+======================   Troubleshooting   ============================
+=======================================================================
+
+The log says the temporary directory is 
+/var/tmp/imapsync_cgi/
+but this directory is not in the system. What a mystery!
+
+It may be that the apache or httpd service is run by systemd with a
+jailed temporary directory.
+
+Solution:
+
+  find /etc/systemd/ /usr/lib/systemd/ | xargs grep -s PrivateTmp
+
+If systemd jails Apache, then you'll find a line like:
+/etc/systemd/system/multi-user.target.wants/apache2.service:PrivateTmp=true
+(Debian/Ubuntu)
+or
+/usr/lib/systemd/system/httpd.service:PrivateTmp=true
+(Centos)
+
+The goal is to override the line
+
+PrivateTmp=true
+
+found in /etc/systemd/system/multi-user.target.wants/apache2.service 
+or
+/usr/lib/systemd/system/httpd.service
+
+with the line:
+
+PrivateTmp=false
+
+The right way to do it is by using the "systemctl edit ..." command
+and then reload the systemd daemon and restart the apache2 service.
+You can also edit directly the file override.conf if you know where
+to do it. If you don't use the override.conf mechanism then your change will 
+be canceled the next time the apache package is updated.
+
+Debian:
+  systemctl edit apache2
+
+  cat /etc/systemd/system/apache2.service.d/override.conf
+[Service]
+PrivateTmp=false
+
+  systemctl daemon-reload
+  systemctl restart apache2
+  systemctl status  apache2
+
+Centos:
+  systemctl edit httpd
+
+  cat /etc/systemd/system/httpd.service.d/override.conf
+[Service]
+PrivateTmp=false
+
+  systemctl daemon-reload
+  systemctl restart httpd
+  systemctl status  httpd
+
+Then retry 
+
+  wget -nv -S -O-  http://localhost/cgi-bin/imapsync?testslive=1
+
+Look now if /var/tmp/imapsync_cgi/ is there.
+
+=======================================================================
+If you encounter this issue:
+
+Failed to find a valid digest in the 'integrity' attribute for resource 
+'https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js' 
+with computed SHA-256 integrity 'kZMXypKF3if9/5v2tP9UHBvS/535tSyH7vjszruyCso='. 
+The resource has been blocked.
+
+It may be because of AdBlock.
+
+Verification:
+
+  wget https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js
+  cat jquery.min.js | openssl dgst -sha384 -binary | openssl base64 -A
+
+gives exactly what is in imapsync_form_extra.html
+
+  more imapsync_form_extra.html
+...
+src="https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js"
+integrity="sha384-xBuQ/xzmlsLoJpyjoggmTEz8OWUFM0/RC5BsqQBDX2v5cMvDHcMakNTNrHIW2I5f"
+
+So if your https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js
+is not what it should be, your access looks compromised.
+
+Thanks to Dominik Ulrich for this insight!
+
+=======================================================================
+=======================================================================
+
+This part is for hackers only.
+
+If you want to use the UI but make it more complicated things than
+just run imapsync then use the following files:
+
+imapsync_shell_wrapper      instead of imapsync itself 
+imapsync_form_wrapper.js    instead of imapsync_form.js
+imapsync_form_wrapper.html  instead of imapsync_form.html
+
+How to get those files:
+
+wget -N https://imapsync.lamiral.info/X/imapsync_shell_wrapper \
+  https://imapsync.lamiral.info/X/imapsync_form_wrapper.js \
+  https://imapsync.lamiral.info/X/imapsync_form_wrapper.html
+
+Centos:
+  chmod +x imapsync_shell_wrapper
+  cp imapsync_shell_wrapper /var/www/cgi-bin/
+
+Debian:
+  chmod +x imapsync_shell_wrapper
+  cp imapsync_shell_wrapper /usr/lib/cgi-bin/
+
+Normally, you only have to change the script imapsync_shell_wrapper to
+suit your needs.
+
+Have in mind that the abort button will kill only one imapsync so it
+is not a working button in case of successive imapsync runs.
+
 =======================================================================
 =======================================================================
 
@@ -223,13 +554,12 @@ That's all for installing a /X service on Centos 7.
 ====== mod_perl failure ======
 
 This part is for mod_perl experts only.
-The script imapsync doesn't work under Modperl::Registry
-nor under ModPerl::PerlRun. So read on if you think you
-are better than me. 
 
-I've try the standard way, telling how any cgi perl script
-can be run under mod_perl perlrun, but it fails with imapsync.
-Any hint welcome!
+The script imapsync doesn't work under Modperl::Registry nor under
+ModPerl::PerlRun. So read on if you think you are better than me.
+
+I tried the standard way, telling how any cgi Perl script can be run
+under mod_perl perlrun, but it fails with imapsync.  Any hint welcome!
 
 # This is a Debian example
 
@@ -268,6 +598,5 @@ Any hint welcome!
 
   curl http://localhost/perl-run/imapsync
 
-
 =======================================================================
 =======================================================================

INSTALL.d/INSTALL.Ubuntu.txt

@@ -1,5 +1,5 @@
 #!/bin/cat 
-# $Id: INSTALL.Ubuntu.txt,v 1.15 2018/11/20 10:28:05 gilles Exp gilles $
+# $Id: INSTALL.Ubuntu.txt,v 1.21 2022/06/03 16:58:47 gilles Exp gilles $
 
 This documentation is also located online at 
 https://imapsync.lamiral.info/INSTALL.d/
@@ -17,20 +17,25 @@ This command installs standard Ubuntu packages:
   sudo apt-get install  \
 libauthen-ntlm-perl     \
 libclass-load-perl      \
+libcrypt-openssl-rsa-perl \
 libcrypt-ssleay-perl    \
 libdata-uniqid-perl     \
 libdigest-hmac-perl     \
 libdist-checkconflicts-perl \
+libencode-imaputf7-perl     \
 libfile-copy-recursive-perl \
+libfile-tail-perl       \
 libio-compress-perl     \
 libio-socket-inet6-perl \
 libio-socket-ssl-perl   \
 libio-tee-perl          \
+libjson-webtoken-perl   \
 libmail-imapclient-perl \
 libmodule-scandeps-perl \
 libnet-dbus-perl        \
 libnet-ssleay-perl      \
 libpar-packer-perl      \
+libproc-processtable-perl \
 libreadonly-perl        \
 libregexp-common-perl   \
 libsys-meminfo-perl     \
@@ -47,6 +52,7 @@ libtest-nowarnings-perl \
 libtest-deep-perl       \
 libtest-warn-perl       \
 make                    \
+time                    \
 cpanminus
 
 In case you want to update the Perl module 
@@ -55,11 +61,6 @@ the following command installs it "manually":
 
   sudo cpanm Mail::IMAPClient
 
-In case you need to use XOAUTH2 authentication you have to install   
-the module JSON::WebToken with the command:
-
-  sudo cpanm JSON::WebToken
-
 After installing the dependencies, imapsync should be working.
 
 You don't have to be root to test and use imapsync.
@@ -82,7 +83,7 @@ A live test showing imapsync job:
 
   ./imapsync --testslive
 
-Now the install command (need root priviledges again):
+Now the install command (need root privileges again):
 
   cp imapsync /usr/bin/
 

INSTALL.d/INSTALL.webserver.txt

@@ -0,0 +1,18 @@
+#!/bin/cat 
+# $Id: INSTALL.webserver.txt,v 1.6 2022/06/15 12:09:39 gilles Exp gilles $
+
+This documentation is also located online at 
+https://imapsync.lamiral.info/INSTALL.d/
+https://imapsync.lamiral.info/INSTALL.d/INSTALL.webserver.txt
+
+
+The script named webserver in the top directory is a small web server
+dedicated to offer the visual interface.  It is experimental and it is
+for now a plain text HTTP server (ie, not https).
+
+
+Dependencies to run webserver on Debian/Ubuntu distributions:
+
+  sudo apt install libnet-server-perl
+
+

INSTALL.d/memo_docker

@@ -1,7 +1,10 @@
 #!/bin/sh
 
+# $Id: memo_docker,v 1.17 2022/04/04 14:46:21 gilles Exp gilles $
+
 echo imapsync_docker_timestamp_dockerfile
-imapsync_docker_timestamp_dockerfile() {
+imapsync_docker_timestamp_dockerfile()
+{
 	DATE_CURRENT=`date +%Y_%m_%d_%H_%M_%S`
 	echo $DATE_CURRENT
 	sed -i -e "/just_a_comment_to_force_update/s/comment_to_force_update.*/comment_to_force_update $DATE_CURRENT/" Dockerfile
@@ -9,38 +12,82 @@ imapsync_docker_timestamp_dockerfile() {
 }
 
 echo imapsync_docker_build
-imapsync_docker_build() {
-    docker build -t gilleslamiral/imapsync .
-    docker images
-    echo 
-    #docker run gilleslamiral/imapsync imapsync --testslive
-    #docker run gilleslamiral/imapsync imapsync --testslive6
-    # docker run gilleslamiral/imapsync imapsync --testslive6 --nossl2
+imapsync_docker_build()
+{
+    docker build -t gilleslamiral/imapsync . \
+        && imapsync_docker_add_tag_version && echo Build SUCCEEDED || echo Build FAILED
 }
 
+echo imapsync_docker_version
+imapsync_docker_version()
+{
+        docker run gilleslamiral/imapsync imapsync --version
+}
+
+echo imapsync_local_docker_version
+imapsync_local_docker_version()
+{
+        docker run gilleslamiral/imapsync /imapsync --version
+}
+
+echo imapsync_docker_add_tag_version
+imapsync_docker_add_tag_version()
+{
+    docker images gilleslamiral/imapsync
+    echo 
+    version=`imapsync_docker_version`
+    echo $version
+    docker tag gilleslamiral/imapsync:latest gilleslamiral/imapsync:$version
+    docker images gilleslamiral/imapsync
+}
+
+
+
 echo imapsync_docker_testslive
-imapsync_docker_testslive() {
-    docker run gilleslamiral/imapsync imapsync --testslive
+imapsync_docker_testslive()
+{
+    echo docker run gilleslamiral/imapsync imapsync --testslive "$@"
+    docker run gilleslamiral/imapsync imapsync --testslive "$@"
 }
 
 echo imapsync_docker_testslive6
-imapsync_docker_testslive6() {
-    docker run gilleslamiral/imapsync imapsync --testslive6 --ssl1 --ssl2
+imapsync_docker_testslive6()
+{
+    echo docker run gilleslamiral/imapsync imapsync --testslive6 "$@"
+    docker run gilleslamiral/imapsync imapsync --testslive6 "$@"
 }
 
 echo imapsync_docker_tests
-imapsync_docker_tests() {
+imapsync_docker_tests()
+{
+    echo docker run gilleslamiral/imapsync imapsync --tests
     docker run gilleslamiral/imapsync imapsync --tests
 }
 
+echo imapsync_docker_envfile
+imapsync_docker_envfile()
+{
+        docker run --env-file ./secret.txt gilleslamiral/imapsync imapsync \
+        --host1 test1.lamiral.info --user1 test1 \
+        --host2 test2.lamiral.info --user2 test2  --justlogin
+}
+
 echo docker_delete_all_images
-docker_delete_all_images() {
+docker_delete_all_images()
+{
     docker rm `docker ps -a -q`
     docker rmi `docker images -q`
 }
 
+echo docker_system_prune
+docker_system_prune()
+{
+        docker system prune
+}
+
 echo docker_delete_dandling_images
-docker_delete_dandling_images() {
+docker_delete_dandling_images()
+{
 	docker images
 	dandling_images=`docker images -f dangling=true -q`
 	exited_containers=`docker ps -a -f status=exited -q`
@@ -51,16 +98,41 @@ docker_delete_dandling_images() {
 
 
 echo imapsync_docker_rebuild_from_scratch
-imapsync_docker_rebuild_from_scratch() {
+imapsync_docker_rebuild_from_scratch()
+{
     delete_all_images
     docker images
     imapsync_docker_build
 }
 
+
+echo is_release_number
+is_release_number()
+{
+        echo "$1" | perl -pe 'm/^\d\.\d+$/ || exit 1' 
+}
+
 echo imapsync_docker_upload
-imapsync_docker_upload() {
-    docker login --username=gilleslamiral --password=`cat $HOME/var/pass/secret.docker` \
-    && docker push gilleslamiral/imapsync
+imapsync_docker_upload()
+{
+        version=`imapsync_docker_version`
+        version_local=`imapsync_local_docker_version`
+        
+        # upload only after successful tests
+        set -x \
+        && is_release_number "$version" \
+        && is_release_number "$version_local" \
+        && test "$version" = "$version_local" \
+        && rcsdiff imapsync Dockerfile memo_docker \
+        && imapsync_docker_testslive \
+        && imapsync_docker_tests \
+        && docker run gilleslamiral/imapsync true \
+        && docker run gilleslamiral/imapsync diff /imapsync /usr/bin/imapsync \
+        && docker login --username=gilleslamiral --password=`cat $HOME/var/pass/secret.docker` \
+        && docker push gilleslamiral/imapsync:latest \
+        && docker push gilleslamiral/imapsync:$version \
+        && set +x && echo imapsync push SUCCEEDED || echo imapsync push FAILED
+        set +x
 }
 
 

INSTALL.d/prerequisites_imapsync

@@ -1,6 +1,6 @@
 #!/bin/sh
 
-# $Id: prerequisites_imapsync,v 1.29 2018/09/28 14:16:44 gilles Exp gilles $
+# $Id: prerequisites_imapsync,v 1.36 2022/05/17 13:46:16 gilles Exp gilles $
 
 MODULES_MANDATORY='
 App::cpanminus
@@ -14,8 +14,11 @@ Digest::HMAC
 Digest::HMAC_MD5
 Digest::MD5
 Dist::CheckConflicts
+Encode
 Encode::Byte
+Encode::IMAPUTF7
 File::Copy::Recursive
+File::Tail
 IO::Socket::INET
 IO::Socket::INET6
 IO::Socket::SSL
@@ -26,6 +29,7 @@ JSON::WebToken::Crypt::RSA
 HTML::Entities
 LWP::UserAgent
 Mail::IMAPClient
+MIME::Base64
 Module::Implementation
 Module::Runtime
 Module::ScanDeps
@@ -35,6 +39,7 @@ Package::Stash::XS
 PAR::Packer
 Parse::RecDescent
 Pod::Usage
+Proc::ProcessTable
 Readonly
 Regexp::Common
 Sys::MemInfo
@@ -46,6 +51,7 @@ Test::More
 Test::Pod
 Test::Requires
 Test::Deep
+Text::ParseWords
 Try::Tiny
 Unicode::String
 URI::Escape
@@ -127,7 +133,7 @@ search_modules_any() {
 	}
         
         # Debian, Ubuntu & Co
-	apt-file -h > /dev/null 2>&1 && {
+	apt-cache -h > /dev/null 2>&1 && {
 		search_modules_apt "$@"
 		return
 	}
@@ -144,7 +150,12 @@ search_modules_any() {
 		return
 	}
         
-	
+        # FreeBSD
+	pkg version > /dev/null 2>&1 && {
+		search_modules_freebsd  "$@"
+		return
+	}
+        
 	# no yum, no apt-get, no pacman
 	{
 		search_modules_cpanm "$@"
@@ -202,16 +213,37 @@ search_modules_apt() {
 	echo 
 	echo Searching deb packages names
 	for M in "$@" ; do
-		echo "==== Searching deb package name for $M"
 		F=`echo $M|tr -s ":" "/"`.pm
+		echo "==== Searching deb package name for $M with: apt-file search /$F"
 		#echo apt-file search /$F
 		echo
-		apt-file search /$F
+                if apt-file -h > /dev/null 2>&1
+                then
+                        apt-file search /$F
+                else
+                        echo "apt-file is not installed. Suggestion: apt-get install apt-file; apt-file update"
+                fi
+                echo
+		echo "==== Searching deb package name for $M with: apt-cache search $M"
 		#apt-cache search "$M"
+                apt-cache search "$M"
 		echo
 	done
 }
 
+search_modules_freebsd()
+{
+	echo 
+	echo Searching pkg FreeBSD names
+	for M in "$@" ; do
+		F=`echo $M|tr -s ":" "-"`
+		echo "==== Searching deb package name for $M with: pkg search p5-$F"
+		echo
+		pkg search "p5-$F"
+                echo
+	done
+
+}
 
 list_to_install() {
 	test -n "$LIST_TO_INSTALL" || return 0
@@ -237,7 +269,7 @@ test_unix() {
 
 test_unix
 #exit
-test_perl || exit
+test_perl || exit 1
 test_make || exit
 test_mandatory_modules
 list_to_install
@@ -246,6 +278,13 @@ EXIT=$?
 # Help the user to install missing modules
 search_modules_any $LIST_TO_INSTALL
 
-test "$1" = "MODULES_MANDATORY" && search_modules_any $MODULES_MANDATORY
+if test "$1" = "MODULES_MANDATORY"
+then
+        search_modules_any $MODULES_MANDATORY
+elif test -n "$1" 
+then
+        search_modules_any "$@"
+fi
+
 test_cpanm
 exit $EXIT

INSTALL.d/secret.txt

@@ -0,0 +1,2 @@
+IMAPSYNC_PASSWORD1=secret1
+IMAPSYNC_PASSWORD2=secret2

README

@@ -1,11 +1,12 @@
 NAME
 
-    imapsync - Email IMAP tool for syncing, copying and migrating email
-    mailboxes between two imap servers, one way, and without duplicates.
+    imapsync - Email IMAP tool for syncing, copying, migrating and archiving
+    email mailboxes between two imap servers, one way, and without
+    duplicates.
 
 VERSION
 
-    This documentation refers to Imapsync $Revision: 1.921 $
+    This documentation refers to Imapsync $Revision: 2.229 $
 
 USAGE
 
@@ -25,45 +26,103 @@ DESCRIPTION
     another.
 
     Imapsync command is a tool allowing incremental and recursive imap
-    transfers from one mailbox to another.
+    transfers from one mailbox to another. If you don't understand the
+    previous sentence, it's normal, it's pedantic computer-oriented jargon.
 
-    By default all folders are transferred, recursively, meaning the whole
-    folder hierarchy is taken, all messages in them, and all messages flags
-    (\Seen \Answered \Flagged etc.) are synced too.
+    All folders are transferred, recursively, meaning the whole folder
+    hierarchy is taken, all messages in them, and all message flags (\Seen
+    \Answered \Flagged etc.) are synced too.
 
     Imapsync reduces the amount of data transferred by not transferring a
-    given message if it resides already on both sides.
+    given message if it already resides on the destination side. Messages
+    that are on the destination side but not on the source side stay as they
+    are. See the --delete2 option to have strict sync and delete them.
 
-    Same specific headers and the transfer is done only once. By default,
-    the identification headers are "Message-Id:" and "Received:" lines but
-    this choice can be changed with the --useheader option.
+    How does imapsync know a message is already on both sides? Same specific
+    headers and the transfer is done only once. By default, the
+    identification headers are "Message-Id:" and "Received:" lines but this
+    choice can be changed with the --useheader option, most often a
+    duplicate problem is solved by using --useheader "Message-Id"
 
     All flags are preserved, unread messages will stay unread, read ones
-    will stay read, deleted will stay deleted.
+    will stay read, deleted will stay deleted. In the IMAP protocol, a
+    deleted message is not deleted, it is marked \Deleted and can be
+    undeleted. Real destruction comes with the EXPUNGE or UIDEXPUNGE IMAP
+    commands.
 
-    You can stop the transfer at any time and restart it later, imapsync
-    works well with bad connections and interruptions, by design.
+    You can abort the transfer at any time and restart it later, imapsync
+    works well with bad connections and interruptions, by design. On a
+    terminal hit Ctr-c twice within two seconds to abort the program. Hit
+    Ctr-c just once makes imapsync reconnect to both imap servers.
 
-    You can decide to delete the messages from the source mailbox after a
-    successful transfer, it can be a good feature when migrating live
-    mailboxes since messages will be only on one side.
+    How do you know the sync is finished and well done? When imapsync ends
+    by itself it mentions it with lines like those:
+
+      Exiting with return value 0 (EX_OK: successful termination) 0/50 nb_errors/max_errors PID 301
+      Removing pidfile /tmp/imapsync.pid
+      Log file is LOG_imapsync/2020_11_17_15_59_22_761_test1_test2.txt ( to change it, use --logfile filepath ; or use --nolog to turn off logging )
+
+    If you don't have those lines it means that either the sync process is
+    still running (or eventually hanging indefinitely) or that it ended
+    without a whisper, a strong kill -9 on Linux for example.
+
+    If you have those final lines then it means the sync process is properly
+    finished. It may have encountered problems though.
+
+    A good synchronization is mentioned by some lines above the last ones,
+    especially those three lines:
+
+      The sync looks good, all 1745 identified messages in host1 are on host2.
+      There is no unidentified message on host1.
+      Detected 0 errors
+
+    Imapsync mentions the total sizes of both accounts at the beginning of
+    the sync and also at the end. Sometimes, even with a strict sync, those
+    total sizes differ, and sometimes they differ a lot. The difference is
+    not a good criterion to conclude the sync went wrong.
+
+    Why? That's because message sizes given by the imap servers are not
+    always accurate, they are not always the same as the actual message
+    sizes of the messages transferred by imapsync. Imapsync use the sizes
+    given by the imap servers to calculate the big total size. They can
+    differ. In the early days, Imapsync used the sizes of the messages as
+    one of the criteria to identify the messages, different sizes implied
+    different messages; but it was a mistake, the same message had different
+    sizes on both sides sometimes, depending on the imap servers.
+
+    Another explanation for a big total size difference is that Gmail
+    doesn't count the size of duplicate messages across folders twice, while
+    imapsync does.
+
+    A classical scenario is synchronizing a mailbox B from another mailbox A
+    where you just want to keep a strict copy of A in B. Strict meaning all
+    messages in A will be in B but no more.
+
+    For a strict synchronization, use the option --delete2. The option
+    --delete2 deletes the messages in the host2 folder B that are not in the
+    host1 folder A. If you also need to destroy host2 folders that are not
+    in host1 then use --delete2folders. See also --delete2foldersonly and
+    --delete2foldersbutnot to set up exceptions on folders to destroy. INBOX
+    will never be destroyed, it's a mandatory folder in IMAP so imapsync
+    doesn't even try to remove it.
+
+    A different scenario is to delete the messages from the source mailbox
+    after a successful transfer, it can be a good feature when migrating
+    mailboxes since messages will be only on one side. The source account
+    will only have messages that are not on the destination yet, ie,
+    messages that arrived after a sync or that failed to be transferred.
 
     In that case, use the --delete1 option. Option --delete1 implies also
-    option --expunge1 so all messages marked deleted on host1 will be really
-    deleted.
+    the option --expunge1 so all messages marked deleted on host1 will be
+    deleted. In IMAP protocol deleting a message does not delete it, it
+    marks it with the flag \Deleted, allowing an undelete. Expunging a
+    folder removes, definitively, all the messages marked as \Deleted in
+    this folder.
 
     You can also decide to remove empty folders once all of their messages
     have been transferred. Add --delete1emptyfolders to obtain this
     behavior.
 
-    A different scenario is synchronizing a mailbox B from another mailbox A
-    in case you just want to keep a "live" copy of A in B.
-
-    For this, option --delete2 has to be used, it deletes messages in host2
-    folder B that are not in host1 folder A. If you also need to destroy
-    host2 folders that are not in host1 then use --delete2folders. See also
-    --delete2foldersonly and --delete2foldersbutnot.
-
     Imapsync is not adequate for maintaining two active imap accounts in
     synchronization when the user plays independently on both sides. Use
     offlineimap (written by John Goerzen) or mbsync (written by Michael R.
@@ -73,45 +132,52 @@ OPTIONS
 
      usage: imapsync [options]
 
-    Mandatory options are the six values, three on each sides, needed to log
-    in into the IMAP servers, ie, a host, a username, and a password, two
-    times.
+    The standard options are the six values forming the credentials. Three
+    values on each side are needed to login into the IMAP servers. These six
+    values are a hostname, a username, and a password, two times.
 
-    Conventions used:
+    Here are the conventions used in the following descriptions of the
+    options:
 
-     str means string
-     int means integer
-     reg means regular expression
-     cmd means command
+     str means a string
+     int means an integer number
+     flo means a float number
+     reg means a regular expression
+     cmd means a command
 
-     --dry               : Makes imapsync doing nothing for real, just print what
+     --dry               : Makes imapsync do nothing for real; it just prints what
                            would be done without --dry.
 
   OPTIONS/credentials
 
-     --host1        str  : Source or "from" imap server. Mandatory.
+     --host1        str  : Source or "from" imap server.
      --port1        int  : Port to connect on host1.
-                           Optional since default port is 143 or 993 if --ssl1
-     --user1        str  : User to login on host1. Mandatory.
-     --password1    str  : Password for the user1.
+                           Optional since default ports are the
+                           well known ports imap/143 or imaps/993.
+     --user1        str  : User to login on host1.
+     --password1    str  : Password of user1.
 
-     --host2        str  : "destination" imap server. Mandatory.
-     --port2        int  : Port to connect on host2.
-                           Optional since default port is 143 or 993 if --ssl2
-     --user2        str  : User to login on host2. Mandatory.
-     --password2    str  : Password for the user2.
+     --host2        str  : "destination" imap server.
+     --port2        int  : Port to connect on host2. Optional
+     --user2        str  : User to login on host2.
+     --password2    str  : Password of user2.
 
      --showpasswords     : Shows passwords on output instead of "MASKED".
-                           Useful to restart a complete run by just reading the log,
-                           or to debug passwords. It's not a secure practice.
+                           Useful to restart a complete run by just reading
+                           the command line used in the log,
+                           or to debug passwords.
+                           It's not a secure practice at all!
 
      --passfile1    str  : Password file for the user1. It must contain the
-                           password on the first line. This option avoids to show
+                           password on the first line. This option avoids showing
                            the password on the command line like --password1 does.
-     --passfile2    str  : Password file for the user2. Contains the password.
+     --passfile2    str  : Password file for the user2.
 
     You can also pass the passwords in the environment variables
-    IMAPSYNC_PASSWORD1 and IMAPSYNC_PASSWORD2
+    IMAPSYNC_PASSWORD1 and IMAPSYNC_PASSWORD2. If you don't pass the user1
+    password via --password1 nor --passfile1 nor $IMAPSYNC_PASSWORD1 then
+    imapsync will prompt to enter the password on the terminal. Same thing
+    for user2 password.
 
   OPTIONS/encryption
 
@@ -136,11 +202,6 @@ OPTIONS
      --sslargs2     str  : Pass any ssl parameter for host2 ssl or tls connection.
                            See --sslargs1
 
-     --timeout1     int  : Connection timeout in seconds for host1.
-                           Default is 120 and 0 means no timeout at all.
-     --timeout2     int  : Connection timeout in seconds for host2.
-                           Default is 120 and 0 means no timeout at all.
-
   OPTIONS/authentication
 
      --authmech1    str  : Auth mechanism to use with host1:
@@ -160,6 +221,29 @@ OPTIONS
      --domain1      str  : Domain on host1 (NTLM authentication).
      --domain2      str  : Domain on host2 (NTLM authentication).
 
+     --oauthaccesstoken1 str : The access token to authenticate with OAUTH2.
+                           It will be combined with the --user1 value to form the 
+                           string to pass with XOAUTH2 authentication.
+                           The password given by --password1 or --passfile1
+                           is ignored but needed on the command line.
+                           Instead of the access token itself, the value can be a
+                           file containing the access token on the first line.
+                           If the value is a file, imapsync reads its first line
+                           and take this line as the access token. The advantage
+                           of the file is that if the access token changes then
+                           imapsync can read it again when it needs to reconnect 
+                           during a run.
+
+
+     --oauthaccesstoken2 str : same thing as --oauthaccesstoken1 
+
+     --oauthdirect1 str  : The direct string to pass with XOAUTH2 authentication.
+                           The password given by --password1 or --passfile1 and 
+                           the user given by --user1 are ignored but they are 
+                           needed to be on the command line. Consider it a bug.
+
+     --oauthdirect2 str  : same thing as oauthdirect1
+
   OPTIONS/folders
 
      --folder       str  : Sync this folder.
@@ -167,15 +251,16 @@ OPTIONS
      --folderrec    str  : Sync this folder recursively.
      --folderrec    str  : and this one, etc.
 
-     --folderfirst  str  : Sync this folder first. --folderfirst "Work"
+     --folderfirst  str  : Sync this folder first. Ex. --folderfirst "INBOX"
      --folderfirst  str  : then this one, etc.
      --folderlast   str  : Sync this folder last. --folderlast "[Gmail]/All Mail"
      --folderlast   str  : then this one, etc.
 
      --nomixfolders      : Do not merge folders when host1 is case-sensitive
                            while host2 is not (like Exchange). Only the first
-                           similar folder is synced (ex: with Sent SENT sent
-                           on host1 only Sent will be synced to host2).
+                           similar folder is synced. Example: with folders
+                           "Sent", "SENT" and "sent" on host1, only "Sent"
+                           will be synced to host2.
 
      --skipemptyfolders  : Empty host1 folders are not created on host2.
 
@@ -193,12 +278,26 @@ OPTIONS
 
      --f1f2    str1=str2 : Force folder str1 to be synced to str2,
                            --f1f2 overrides --automap and --regextrans2.
+                           Use several --f1f2 options to map several folders.
+                           Option --f1f2 is a one to one only folder mapping,
+                           str1 and str2 have to be full path folder names.
 
      --subfolder2   str  : Syncs the whole host1 folders hierarchy under the
                            host2 folder named str.
-                           (It does it internally by adding two
+                           It does it internally by adding three
                            --regextrans2 options before all others.
-                           Add --debug to see what's really going on.)
+                           Add --debug to see what's really going on.
+
+     --subfolder1   str  : Syncs the host1 folders hierarchy which is under folder
+                           str to the root hierarchy of host2.
+                           It's the counterpart of a sync done by --subfolder2
+                           when doing it in the reverse order.
+                           Backup/Restore scenario:
+                           Use --subfolder2 str for a backup to the folder str
+                           on host2. Then use --subfolder1 str for restoring
+                           from the folder str, after inverting
+                           host1/host2 user1/user2 values.
+
 
      --subscribed        : Transfers subscribed folders.
      --subscribe         : Subscribe to the folders transferred on the
@@ -207,21 +306,30 @@ OPTIONS
                            host2 even if they are not subscribed on host1.
 
      --prefix1      str  : Remove prefix str to all destination folders,
-                           usually INBOX. or INBOX/ or an empty string "".
+                           usually "INBOX." or "INBOX/" or an empty string "".
                            imapsync guesses the prefix if host1 imap server
-                           does not have NAMESPACE capability. This option
-                           should not be used, most of the time.
+                           does not have NAMESPACE capability. So this option
+                           should not be used most of the time.
      --prefix2      str  : Add prefix to all host2 folders. See --prefix1
-     --sep1         str  : Host1 separator in case NAMESPACE is not supported.
-     --sep2         str  : Host2 separator in case NAMESPACE is not supported.
+
+     --sep1         str  : Host1 separator. This option should not be used
+                           most of the time.
+                           Imapsync gets the separator from the server itself,
+                           by using NAMESPACE, or it tries to guess it
+                           from the folders listing (it counts
+                           characters / . \\ \ in folder names and choose the
+                           more frequent, or finally / if nothing is found.
+     --sep2         str  : Host2 separator. See --sep1
 
      --regextrans2  reg  : Apply the whole regex to each destination folders.
      --regextrans2  reg  : and this one. etc.
                            When you play with the --regextrans2 option, first
                            add also the safe options --dry --justfolders
-                           Then, when happy, remove --dry, remove --justfolders.
-                           Have in mind that --regextrans2 is applied after prefix
-                           and separator inversion. For examples see
+                           Then, when happy, remove --dry for a run, then
+                           remove --justfolders for the next ones.
+                           Have in mind that --regextrans2 is applied after
+                           the automatic prefix and separator inversion.
+                           For examples see:
                            https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Mapping.txt
 
   OPTIONS/folders sizes
@@ -239,9 +347,11 @@ OPTIONS
                            Default is system specific, Unix is /tmp but
                            /tmp is often too small and deleted at reboot.
                            --tmpdir /var/tmp should be better.
+
      --pidfile      str  : The file where imapsync pid is written,
-                           it can be dirname/filename.
-                           Default name is imapsync.pid in tmpdir.
+                           it can be dirname/filename complete path.
+                           The default name is imapsync.pid in tmpdir.
+
      --pidfilelocking    : Abort if pidfile already exists. Useful to avoid
                            concurrent transfers on the same mailbox.
 
@@ -251,25 +361,80 @@ OPTIONS
      --logfile      str  : Change the default log filename (can be dirname/filename).
      --logdir       str  : Change the default log directory. Default is LOG_imapsync/
 
+    The default logfile name is for example
+
+     LOG_imapsync/2019_12_22_23_57_59_532_user1_user2.txt
+
+    where:
+
+     2019_12_22_23_57_59_532 is nearly the date of the start
+     YYYY_MM_DD_HH_MM_SS_mmm
+     year_month_day_hour_minute_second_millisecond
+
+    and user1 user2 are the --user1 --user2 values.
+
   OPTIONS/messages
 
      --skipmess     reg  : Skips messages matching the regex.
-                           Example: 'm/[\x80-ff]/' # to avoid 8bits messages.
+                           Example: 'm/[\x80-\xff]/' # to avoid 8bits messages.
                            --skipmess is applied before --regexmess
      --skipmess     reg  : or this one, etc.
 
+     --skipcrossduplicates : Avoid copying messages that are already copied
+                             in another folder, good from Gmail to XYZ when
+                             XYZ is not also Gmail.
+                             Activated with --gmail1 unless --noskipcrossduplicates
+
+     --debugcrossduplicates : Prints which messages (UIDs) are skipped with
+                              --skipcrossduplicates and in what other folders
+                              they are.
+
      --pipemess     cmd  : Apply this cmd command to each message content
                            before the copy.
      --pipemess     cmd  : and this one, etc.
+                           With several --pipemess, the output of each cmd
+                           command (STDOUT) is given to the input (STDIN)
+                           of the next command.
+                           For example,
+                           --pipemess cmd1 --pipemess cmd2 --pipemess cmd3
+                           is like a Unix pipe:
+                           "cat message | cmd1 | cmd2 | cmd3"
 
      --disarmreadreceipts : Disarms read receipts (host2 Exchange issue)
 
      --regexmess    reg  : Apply the whole regex to each message before transfer.
-                           Example: 's/\000/ /g' # to replace null by space.
+                           Example: 's/\000/ /g' # to replace null characters
+                           by spaces.
      --regexmess    reg  : and this one, etc.
+ 
+     --truncmess    int  : truncates messages when their size exceed the int 
+                           value, specified in bytes. Good to sync too big 
+                           messages or to "suppress" attachments. 
+                           Have in mind that this way, messages become 
+                           uncoherent somehow.
+
+  OPTIONS/labels
+
+    Gmail present labels as folders in imap. Imapsync can accelerate the
+    sync by syncing X-GM-LABELS, it will avoid to transfer messages when
+    they are already on host2 in another folder.
+
+     --synclabels        : Syncs also Gmail labels when a message is copied to host2.
+                           Activated by default with --gmail1 --gmail2 unless
+                           --nosynclabels is added.
+
+     --resynclabels      : Resyncs Gmail labels when a message is already on host2.
+                           Activated by default with --gmail1 --gmail2 unless
+                           --noresynclabels is added.
+
+    For Gmail syncs, see also:
+    https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt
 
   OPTIONS/flags
 
+     If you encounter flag problems see also:
+     https://imapsync.lamiral.info/FAQ.d/FAQ.Flags.txt
+
      --regexflag    reg  : Apply the whole regex to each flags list.
                            Example: 's/"Junk"//g' # to remove "Junk" flag.
      --regexflag    reg  : then this one, etc.
@@ -280,6 +445,9 @@ OPTIONS
                            May be useful when a user has already started to play
                            with its host2 account.
 
+     --filterbuggyflags  : Filter flags known to be buggy and generators of errors
+                           "BAD Invalid system flag" or "NO APPEND Invalid flag list".
+
   OPTIONS/deletions
 
      --delete1           : Deletes messages on host1 server after a successful
@@ -288,55 +456,71 @@ OPTIONS
                            \Deleted, then messages are really deleted with an
                            EXPUNGE IMAP command. If expunging after each message
                            slows down too much the sync then use
-                           --noexpungeaftereach to speed up.
+                           --noexpungeaftereach to speed up, expunging will then be
+                           done only twice per folder, one at the beginning and
+                           one at the end of a folder sync.
+
      --expunge1          : Expunge messages on host1 just before syncing a folder.
                            Expunge is done per folder.
                            Expunge aims is to really delete messages marked deleted.
                            An expunge is also done after each message copied
-                           if option --delete1 is set.
+                           if option --delete1 is set (unless --noexpungeaftereach).
+
      --noexpunge1        : Do not expunge messages on host1.
+
      --delete1emptyfolders : Deletes empty folders on host1, INBOX excepted.
                              Useful with --delete1 since what remains on host1
                              is only what failed to be synced.
 
-     --delete2           : Delete messages in host2 that are not in
-                           host1 server. Useful for backup or pre-sync.
-     --delete2duplicates : Delete messages in host2 that are duplicates.
+     --delete2           : Delete messages in the host2 account that are not in
+                           the host1 account. Useful for backup or pre-sync.
+                           --delete2 implies --uidexpunge2
+
+     --delete2duplicates : Deletes messages in host2 that are duplicates in host2.
                            Works only without --useuid since duplicates are
                            detected with an header part of each message.
+                           NB: --delete2duplicates is far less violent than --delete2
+                           since it removes only duplicates.
 
-     --delete2folders    : Delete folders in host2 that are not in host1 server.
-                           For safety, first try it like this (it is safe):
+     --delete2folders    : Delete folders in host2 that are not in host1.
+                           For safety, first try it like this, it is safe:
                            --delete2folders --dry --justfolders --nofoldersizes
-     --delete2foldersonly   reg : Deleted only folders matching regex.
-                                  Example: --delete2foldersonly "/^Junk$|^INBOX.Junk$/"
-     --delete2foldersbutnot reg : Do not delete folders matching regex.
-                                  Example: --delete2foldersbutnot "/Tasks$|Contacts$|Foo$/"
+                           and see what folders will be deleted.
 
-     --expunge2          : Expunge messages on host2 after messages transfer.
-     --uidexpunge2       : uidexpunge messages on the host2 account
-                           that are not on the host1 account, requires --delete2
+     --delete2foldersonly   reg : Delete only folders matching the regex reg.
+                                  Example: --delete2foldersonly "/^Junk$|^INBOX.Junk$/"
+                                  This option activates --delete2folders
+
+     --delete2foldersbutnot reg : Do not delete folders matching the regex rex.
+                                  Example: --delete2foldersbutnot "/Tasks$|Contacts$|Foo$/"
+                                  This option activates --delete2folders
+
+     --noexpunge2        : Do not expunge messages on host2.
+     --nouidexpunge2     : Do not uidexpunge messages on the host2 account
+                           that are not on the host1 account.
 
   OPTIONS/dates
 
-     --syncinternaldates : Sets the internal dates on host2 same as host1.
+     If you encounter problems with dates, see also:
+     https://imapsync.lamiral.info/FAQ.d/FAQ.Dates.txt
+
+     --syncinternaldates : Sets the internal dates on host2 as the same as host1.
                            Turned on by default. Internal date is the date
-                           a message arrived on a host (mtime).
-     --idatefromheader   : Sets the internal dates on host2 same as the
-                           "Date:" headers.
-                           If you encounter problems with dates see also
-                           https://imapsync.lamiral.info/FAQ.d/FAQ.Dates.txt
+                           a message arrived on a host (Unix mtime usually).
+     --idatefromheader   : Sets the internal dates on host2 as same as the
+                           ones in "Date:" headers.
 
   OPTIONS/message selection
 
      --maxsize      int  : Skip messages larger  (or equal) than  int  bytes
      --minsize      int  : Skip messages smaller (or equal) than  int  bytes
-     --maxage       int  : Skip messages older than  int  days.
+
+     --maxage       int  : Skip messages older than  int days.
                            final stats (skipped) don't count older messages
                            see also --minage
      --minage       int  : Skip messages newer than  int  days.
                            final stats (skipped) don't count newer messages
-                           You can do (+ are the messages selected):
+                           You can do (+ zone are the messages selected):
                            past|----maxage+++++++++++++++>now
                            past|+++++++++++++++minage---->now
                            past|----maxage+++++minage---->now (intersection)
@@ -344,37 +528,54 @@ OPTIONS
 
      --search       str  : Selects only messages returned by this IMAP SEARCH
                            command. Applied on both sides.
-                           For a complete of what can be search see
+                           For a complete set of what can be search see
                            https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt
 
      --search1      str  : Same as --search but for selecting host1 messages only.
      --search2      str  : Same as --search but for selecting host2 messages only.
-                           --search CRIT equals --search1 CRIT --search2 CRIT
+                           So --search CRIT equals --search1 CRIT --search2 CRIT
+
+     --noabletosearch    : Makes --minage and --maxage options use the internal
+                           dates given by a FETCH imap command instead of the
+                           "Date:" header. Internal date is the arrival date
+                           in the mailbox.
+                           --noabletosearch equals --noabletosearch1 --noabletosearch2
+
+     --noabletosearch1   : Like --noabletosearch but for host1 only.
+     --noabletosearch2   : Like --noabletosearch but for host2 only.
 
      --maxlinelength int : skip messages with a line length longer than  int  bytes.
-                           RFC 2822 says it must be no more than 1000 bytes.
+                           RFC 2822 says it must be no more than 1000 bytes but
+                           real life servers and email clients do more.
 
 
      --useheader    str  : Use this header to compare messages on both sides.
-                           Ex: Message-ID or Subject or Date.
+                           Example: "Message-Id" or "Received" or "Date".
      --useheader    str    and this one, etc.
 
-     --usecache          : Use cache to speed up the sync.
+     --syncduplicates    : Sync also duplicates. Off by default.
+
+     --usecache          : Use cache to speed up next syncs. Off by default.
      --nousecache        : Do not use cache. Caveat: --useuid --nousecache creates
                            duplicates on multiple runs.
-     --useuid            : Use UIDs instead of headers as a criterium to recognize
+
+     --useuid            : Use UIDs instead of headers as a criterion to recognize
                            messages. Option --usecache is then implied unless
                            --nousecache is used.
 
   OPTIONS/miscellaneous
 
      --syncacls          : Synchronizes acls (Access Control Lists).
+                           Acls in IMAP are not standardized, be careful
+                           since one acl code on one side may signify something
+                           else on the other one.
      --nosyncacls        : Does not synchronize acls. This is the default.
-                           Acls in IMAP are not standardized, be careful.
-     --addheader         : When a message has no headers to be identified, 
+
+     --addheader         : When a message has no headers to be identified,
                            --addheader adds a "Message-Id" header,
                            like "Message-Id: 12345@imapsync", where 12345
                            is the imap UID of the message on the host1 folder.
+                           Useful to sync folders "Sent" or "Draft".
 
   OPTIONS/debugging
 
@@ -392,47 +593,83 @@ OPTIONS
      --tests             : Run local non-regression tests. Exit code 0 means all ok.
      --testslive         : Run a live test with test1.lamiral.info imap server.
                            Useful to check the basics. Needs internet connection.
-     --testslive6        : Run a live test with ks2ipv6.lamiral.info imap server.
+     --testslive6        : Run a live test with ks6ipv6.lamiral.info imap server.
                            Useful to check the ipv6 connectivity. Needs internet.
 
   OPTIONS/specific
 
-      --gmail1           : sets --host1 to Gmail and options from FAQ.Gmail.txt
-      --gmail2           : sets --host2 to Gmail and options from FAQ.Gmail.txt
+      --gmail1           : sets --host1 to Gmail and other options. See FAQ.Gmail.txt
+      --gmail2           : sets --host2 to Gmail and other options. See FAQ.Gmail.txt
 
-      --office1          : sets --host1 to Office365 options from FAQ.Exchange.txt
-      --office2          : sets --host2 to Office365 options from FAQ.Exchange.txt
+      --office1          : sets --host1 to Office365 and other options. See FAQ.Office365.txt
+      --office2          : sets --host2 to Office365 and other options. See FAQ.Office365.txt
 
-      --exchange1        : sets options from FAQ.Exchange.txt, account1 part
-      --exchange2        : sets options from FAQ.Exchange.txt, account2 part
+      --exchange1        : sets options for Exchange. See FAQ.Exchange.txt
+      --exchange2        : sets options for Exchange. See FAQ.Exchange.txt
 
-      --domino1          : sets options from FAQ.Domino.txt, account1 part
-      --domino2          : sets options from FAQ.Domino.txt, account2 part
+      --domino1          : sets options for Domino. See FAQ.Domino.txt
+      --domino2          : sets options for Domino. See FAQ.Domino.txt
 
   OPTIONS/behavior
 
-     --maxmessagespersecond int : limits the number of messages transferred per second.
+     --timeout1     flo  : Connection timeout in seconds for host1.
+                           Default is 120 and 0 means no timeout at all.
+     --timeout2     flo  : Connection timeout in seconds for host2.
+                           Default is 120 and 0 means no timeout at all.
+
+                           Caveat, under CGI context, you may encounter a timeout
+                           from the webserver, killing imapsync and the imap connections.
+                           See the document INSTALL.OnlineUI.txt and search
+                           for "Timeout" for how to deal with this issue.
+
+     --keepalive1        : https://metacpan.org/pod/Mail::IMAPClient#Keepalive
+                           Some firewalls and network gears like to timeout connections 
+                           prematurely if the connection sits idle.
+                           This option enables SO_KEEPALIVE on the host1 socket.
+                           --keepalive1 is on by default since imapsync release 2.169
+                           Use --nokeepalive1 to disable it.
+                       
+     --keepalive2        : Same as --keepalive2 but for host2.
+                           Use --nokeepalive2 to disable it.
+
+     --maxmessagespersecond flo : limits the average number of messages 
+                                  transferred per second.
 
      --maxbytespersecond int : limits the average transfer rate per second.
      --maxbytesafter     int : starts --maxbytespersecond limitation only after
                                --maxbytesafter amount of data transferred.
 
-     --maxsleep      int : do not sleep more than int seconds.
+     --maxsleep      flo : do not sleep more than int seconds.
                            On by default, 2 seconds max, like --maxsleep 2
 
      --abort             : terminates a previous call still running.
                            It uses the pidfile to know what process to abort.
 
-     --exitwhenover int  : Stop syncing when total bytes transferred reached.
+     --exitwhenover int  : Stop syncing and exits when int total bytes
+                           transferred is reached.
+
+     --version           : Print only the software version.
+     --noreleasecheck    : Do not check for any new imapsync release.
+     --releasecheck      : Check for new imapsync release.
+                           it's an http request to
+                           http://imapsync.lamiral.info/prj/imapsync/VERSION
+
+      --emailreport1     : Put the email final report in host1 INBOX
+      --emailreport2     : Put the email final report in host2 INBOX
+
+      --noemailreport1   : Do not put the email final report in host1 INBOX
+      --noemailreport2   : Do not put the email final report in host2 INBOX
+
+     --noid              : Do not send/receive IMAP "ID" command to imap servers.
 
-     --version           : Print only software version.
-     --noreleasecheck    : Do not check for new imapsync release (a http request).
-     --releasecheck      : Check for new imapsync release (a http request).
-     --noid              : Do not send/receive ID command to imap servers.
      --justconnect       : Just connect to both servers and print useful
                            information. Need only --host1 and --host2 options.
+                           Obsolete since "imapsync --host1 imaphost" alone
+                           implies --justconnect
+
      --justlogin         : Just login to both host1 and host2 with users
                            credentials, then exit.
+
      --justfolders       : Do only things about folders (ignore messages).
 
      --help              : print this help.
@@ -448,22 +685,26 @@ OPTIONS
 
 SECURITY
 
-    You can use --passfile1 instead of --password1 to give the password
-    since it is safer. With --password1 option, any user on your host can
-    see the password by using the 'ps auxwwww' command. Using a variable
-    (like $PASSWORD1) is also dangerous because of the 'ps auxwwwwe'
-    command. So, saving the password in a well protected file (600 or
-    rw-------) is the best solution.
+    You can use --passfile1 instead of --password1 to mention the password
+    since it is safer. With --password1 option, on Linux, any user on your
+    host can see the password by using the 'ps auxwwww' command. Using a
+    variable (like IMAPSYNC_PASSWORD1) is also dangerous because of the 'ps
+    auxwwwwe' command. So, saving the password in a well protected file (600
+    or rw-------) is the best solution.
 
-    Imapsync activates ssl or tls encryption by default, if possible. What
-    detailed behavior is under this "if possible"? Imapsync activates ssl if
-    the well known port imaps port (993) is open on the imap servers. If the
-    imaps port is closed then it open a normal (clear) connection on port
-    143 but it looks for TLS support in the CAPABILITY list of the servers.
-    If TLS is supported then imapsync goes to encryption.
+    Imapsync activates ssl or tls encryption by default, if possible.
 
-    If the automatic ssl/tls detection fails then imapsync will not protect
-    against sniffing activities on the network, especially for passwords.
+    What detailed behavior is under this "if possible"?
+
+    Imapsync activates ssl if the well known port imaps port (993) is open
+    on the imap servers. If the imaps port is closed then it open a normal
+    (clear) connection on port 143 but it looks for TLS support in the
+    CAPABILITY list of the servers. If TLS is supported then imapsync goes
+    to encryption with STARTTLS.
+
+    If the automatic ssl and the tls detections fail then imapsync will not
+    protect against sniffing activities on the network, especially for
+    passwords.
 
     If you want to force ssl or tls just use --ssl1 --ssl2 or --tls1 --tls2
 
@@ -473,14 +714,47 @@ SECURITY
 EXIT STATUS
 
     Imapsync will exit with a 0 status (return code) if everything went
-    good. Otherwise, it exits with a non-zero status.
+    good. Otherwise, it exits with a non-zero status. That's classical Unix
+    behavior. Here is the list of the exit code values (an integer between 0
+    and 255). In Bourne Shells, this exit code value can be retrieved within
+    the variable value "$?" if you read it just after the imapsync call.
+
+    The names reflect their meaning:
+
+         EX_OK          => 0  ; #/* successful termination */
+         EX_USAGE       => 64 ; #/* command line usage error */
+         EX_NOINPUT     => 66 ; #/* cannot open input */
+         EX_UNAVAILABLE => 69 ; #/* service unavailable */
+         EX_SOFTWARE    => 70 ; #/* internal software error */
+         EXIT_CATCH_ALL              =>   1 ; # Any other error
+         EXIT_BY_SIGNAL              =>   6 ; # Should be 128+n where n is the sig_num
+         EXIT_BY_FILE                =>   7 ;
+         EXIT_PID_FILE_ERROR         =>   8 ;
+         EXIT_CONNECTION_FAILURE     =>  10 ;
+         EXIT_TLS_FAILURE            =>  12 ;
+         EXIT_AUTHENTICATION_FAILURE =>  16 ;
+         EXIT_SUBFOLDER1_NO_EXISTS   =>  21 ;
+         EXIT_WITH_ERRORS            => 111 ;
+         EXIT_WITH_ERRORS_MAX        => 112 ;
+         EXIT_OVERQUOTA              => 113 ;
+         EXIT_ERR_APPEND             => 114 ;
+         EXIT_ERR_FETCH              => 115 ;
+         EXIT_ERR_CREATE             => 116 ;
+         EXIT_ERR_SELECT             => 117 ;
+         EXIT_TRANSFER_EXCEEDED      => 118 ;
+         EXIT_ERR_APPEND_VIRUS       => 119 ;
+         EXIT_TESTS_FAILED           => 254 ; # Like Test::More API
+         EXIT_CONNECTION_FAILURE_HOST1     =>  101 ;
+         EXIT_CONNECTION_FAILURE_HOST2     =>  102 ;
+         EXIT_AUTHENTICATION_FAILURE_USER1 =>  161 ;
+         EXIT_AUTHENTICATION_FAILURE_USER2 =>  162 ;
 
 LICENSE AND COPYRIGHT
 
     Imapsync is free, open, public but not always gratis software cover by
-    the NOLIMIT Public License. See the LICENSE file included in the
-    distribution or just read this simple sentence as it IS the licence
-    text:
+    the NOLIMIT Public License, now called NLPL. See the LICENSE file
+    included in the distribution or just read the following simple sentence
+    as it IS the license text:
 
      "No limits to do anything with this work and this license."
 
@@ -488,19 +762,19 @@ LICENSE AND COPYRIGHT
 
      "No limits to do anything with this work and this license."
 
-    https://imapsync.lamiral.info/LICENSE
+    Look at https://imapsync.lamiral.info/LICENSE
 
 AUTHOR
 
     Gilles LAMIRAL <gilles@lamiral.info>
 
-    Feedback good or bad is very often welcome.
+    Good feedback is always welcome. Bad feedback is very often welcome.
 
     Gilles LAMIRAL earns his living by writing, installing, configuring and
-    teaching free, open and often gratis software. Imapsync used to be
-    "always gratis" but now it is only "often gratis" because imapsync is
-    sold by its author, a good way to maintain and support free open public
-    software over decades.
+    sometimes teaching free, open, and often gratis software. Imapsync used
+    to be "always gratis" but now it is only "often gratis" because imapsync
+    is sold by its author, your servitor, a good way to maintain and support
+    free open public software tools over decades.
 
 BUGS AND LIMITATIONS
 
@@ -512,10 +786,6 @@ IMAP SERVERS supported
 
 HUGE MIGRATION
 
-    Pay special attention to options --subscribed --subscribe --delete1
-    --delete1emptyfolders --delete2 --delete2folders --maxage --minage
-    --maxsize --useuid --usecache
-
     If you have many mailboxes to migrate think about a little shell
     program. Write a file called file.txt (for example) containing users and
     passwords. The separator used in this example is ';'
@@ -549,23 +819,23 @@ HUGE MIGRATION
 
 INSTALL
 
-     Imapsync works under any Unix with perl.
+     Imapsync works under any Unix with Perl.
 
      Imapsync works under most Windows (2000, XP, Vista, Seven, Eight, Ten
-     and all Server releases 2000, 2003, 2008 and R2, 2012 and R2)
-     as a standalone binary software called imapsync.exe,
-     usually launched from a batch file in order to avoid always typing
-     the options.
+     and all Server releases 2000, 2003, 2008 and R2, 2012 and R2, 2016)
+     as a standalone binary software called imapsync.exe, usually launched
+     from a batch file in order to avoid always typing the options. There
+     is also a 32bit binary called imapsync_32bit.exe
 
-     Imapsync works under OS X as a standalone binary
-     software called imapsync_bin_Darwin
+     Imapsync works under OS X as a standalone binary software called
+     imapsync_bin_Darwin
 
      Purchase latest imapsync at
      https://imapsync.lamiral.info/
 
-     You'll receive a link to a compressed tarball called imapsync-x.xx.tgz
-     where x.xx is the version number. Untar the tarball where
-     you want (on Unix):
+     You'll receive a link to a compressed tarball called
+     imapsync-x.xx.tgz where x.xx is the version number. 
+     Untar the tarball where you want (on Unix):
 
       tar xzvf  imapsync-x.xx.tgz
 
@@ -590,37 +860,56 @@ SIMILAR SOFTWARE
       See also https://imapsync.lamiral.info/S/external.shtml
       for a better up to date list.
 
-      imap_tools    : https://github.com/andrewnimmo/rick-sanders-imap-tools
-      offlineimap   : https://github.com/nicolas33/offlineimap
-      Doveadm-Sync  : http://wiki2.dovecot.org/Tools/Doveadm/Sync
-                      ( Dovecot sync tool )
-      mbsync        : http://isync.sourceforge.net/
-      mailsync      : http://mailsync.sourceforge.net/
-      mailutil      : http://www.washington.edu/imap/
-                      part of the UW IMAP tookit.
-      imaprepl      : http://www.bl0rg.net/software/
-                      http://freecode.com/projects/imap-repl/
-      imapcopy      : http://www.ardiehl.de/imapcopy/
-      migrationtool : http://sourceforge.net/projects/migrationtool/
-      imapmigrate   : http://sourceforge.net/projects/cyrus-utils/
-      wonko_imapsync: http://wonko.com/article/554
-                      see also file W/tools/wonko_ruby_imapsync
-      exchange-away : http://exchange-away.sourceforge.net/
-      pop2imap      : http://www.linux-france.org/prj/pop2imap/
+    List verified on Friday July 1, 2021.
 
-    Feedback (good or bad) will often be welcome.
+     imapsync: https://github.com/imapsync/imapsync (this is an imapsync copy, sometimes delayed, with --noreleasecheck by default since release 1.592, 2014/05/22)
+     imap_tools: https://web.archive.org/web/20161228145952/http://www.athensfbc.com/imap_tools/. The imap_tools code is now at https://github.com/andrewnimmo/rick-sanders-imap-tools
+     imaputils: https://github.com/mtsatsenko/imaputils (very old imap_tools fork)
+     Doveadm-Sync: https://wiki2.dovecot.org/Tools/Doveadm/Sync ( Dovecot sync tool )
+     davmail: http://davmail.sourceforge.net/
+     offlineimap: http://offlineimap.org/
+     fdm: https://github.com/nicm/fdm
+     mbsync: http://isync.sourceforge.net/
+     mailsync: http://mailsync.sourceforge.net/
+     mailutil: https://www.washington.edu/imap/ part of the UW IMAP toolkit. (well, seems abandoned now)
+     imaprepl: https://bl0rg.net/software/ http://freecode.com/projects/imap-repl/
+     imapcopy (Pascal): http://www.ardiehl.de/imapcopy/
+     imapcopy (Java): https://code.google.com/archive/p/imapcopy/
+     imapsize: http://www.broobles.com/imapsize/
+     migrationtool: http://sourceforge.net/projects/migrationtool/
+     imapmigrate: http://sourceforge.net/projects/cyrus-utils/
+     larch: https://github.com/rgrove/larch (derived from wonko_imapsync, good at Gmail)
+     wonko_imapsync: http://wonko.com/article/554 (superseded by larch)
+     pop2imap: https://pop2imap.lamiral.info/ (I wrote that too)
+     exchange-away: http://exchange-away.sourceforge.net/
+     SyncBackPro: http://www.2brightsparks.com/syncback/sbpro.html
+     ImapSyncClient: https://github.com/ridaamirini/ImapSyncClient
+     MailStore: https://www.mailstore.com/en/products/mailstore-home/
+     mnIMAPSync: https://github.com/manusa/mnIMAPSync
+     imap-upload: http://imap-upload.sourceforge.net/ (A tool for uploading a local mbox file to IMAP4 server)
+     imapbackup: https://github.com/rcarmo/imapbackup (A Python script for incremental backups of IMAP mailboxes)
+     BitRecover email-backup 99 USD, 299 USD https://www.bitrecover.com/email-backup/.
+     ImportExportTools: https://addons.thunderbird.net/en-us/thunderbird/addon/importexporttools/ ImportExportTools for Mozilla Thunderbird by Paolo Kaosmos. ImportExportTools does not do IMAP.
+     rximapmail: https://sourceforge.net/projects/rximapmail/
+     Shoviv: https://www.shoviv.com/email-backup.html Allows backup from various sources including Office 365 (Exchange Online, Exchange On-Premises), IMAP, and Google Workspace.
+     CodeTwo: https://www.codetwo.com/ but CodeTwo does imap source to Office365 only.
 
 HISTORY
 
-    I wrote imapsync because an enterprise (basystemes) paid me to install a
-    new imap server without losing huge old mailboxes located in a far away
-    remote imap server, accessible by a low-bandwidth often broken link. The
-    tool imapcp (written in python) could not help me because I had to
-    verify every mailbox was well transferred, and then delete it after a
-    good transfer. Imapsync started its life as a patch of the
-    copy_folder.pl script. The script copy_folder.pl comes from the
-    Mail-IMAPClient-2.1.3 perl module tarball source (more precisely in the
-    examples/ directory of the Mail-IMAPClient tarball). So many happened
-    since then that I wonder if it remains any lines of the original
-    copy_folder.pl in imapsync source code.
+    I initially wrote imapsync in July 2001 because an enterprise, called
+    BaSystemes, paid me to install a new imap server without losing huge old
+    mailboxes located in a far away remote imap server, accessible by an
+    often broken low-bandwidth ISDN link.
+
+    I had to verify every mailbox was well transferred, all folders, all
+    messages, without wasting bandwidth or creating duplicates upon resyncs.
+    The imapsync design was made with the beautiful rsync command in mind.
+
+    Imapsync started its life as a patch of the copy_folder.pl script. The
+    script copy_folder.pl comes from the Mail-IMAPClient-2.1.3 perl module
+    tarball source (more precisely in the examples/ directory of the
+    Mail-IMAPClient tarball).
+
+    So many changes happened since then that I wonder if it remains any
+    lines of the original copy_folder.pl in imapsync source code.
 

README.txt

@@ -0,0 +1 @@
+README

README_Windows.txt

@@ -1,115 +1,225 @@
-# $Id: README_Windows.txt,v 1.13 2019/02/14 15:50:09 gilles Exp gilles $
+# $Id: README_Windows.txt,v 1.27 2022/04/05 19:45:21 gilles Exp gilles $
 # 
 # This is the README_Windows.txt file for imapsync 
-# imapsync : IMAP sync and migrate tool.
+# imapsync: IMAP syncing and migration tool.
 
-WINDOWS
-=======
+=====================
+ Imapsync on Windows
+=====================
 
-There is two ways to install and use imapsync on Windows systems: A) or B).
+ There are two ways to install and use imapsync on Windows systems: A) or B).
 
-Standard users should take the A) way, the simplest way.
+ Standard users should take the A) way, the simplest way.
 
-Developers, or powerful users that want to build their own imapsync.exe
-or modify it, have to consider the B) way, the complex but powerful way.
+ Developers, or power users that want to build their own imapsync.exe
+ or modify it, have to consider the B) way, the complex and powerful way.
 
+
+---------------
 A) Simplest way
 ---------------
 
+A.0) Preamble for visual users looking for a visual tool.
+
+ First, the visual thing I am talking about here is not the online
+ visual interface I call /X that you might have seen and used at the
+ URL https://imapsync.lamiral.info/X/ 
+
+ This /X visual interface is not yet available on Windows as a service
+ that you can install and use in your system.  I plan to make a visual
+ tool available on Windows but, for now, I encountered technical
+ issues.
+
+ So what visual thing am I talking about?
+ 
+ Imapsync itself is not a visual tool. The visual tool is Notepad or
+ any text editor. Many pure visual users have succeeded in using
+ imapsync to transfer their email accounts. As you can guess, only the
+ ones that tried have succeeded, so don't give up before trying at
+ least once. Another assumption is that visual users can read.
+ 
+ Let's go for some reading!
+
 A.1) Get imapsync.
 
-Get imapsync at https://imapsync.lamiral.info/
-You'll then have access to a zip archive file named imapsync_1.xxx.zip
-where 1.xxx is the imapsync release number.
+ Get imapsync at https://imapsync.lamiral.info/
+ You'll then have access to a zip archive file named imapsync_2.200.zip
+ where 2.200 is the imapsync release number.
 
 A.2) Extract the zip file in a folder where you will work with imapsync.
 
-You can work on the Desktop since the zip file extraction creates
-a unique folder named imapsync_1.xxx (where 1.xxx is the imapsync 
-release number).
+ You can work on the Desktop since the zip file extraction creates
+ a unique folder named imapsync_2.200/
+ 
+ Do not unzip the archive in what is called a "system" directory since
+ you may encounter permission issues.
+
+Two points to have in mind:
+* You don't need to be an Administrator to unzip the zip archive.
+* You don't need to be an Administrator to run imapsync.
+
+In short, any user on your system can use imapsync.
 
 A.3) Check the folder
 
-In the folder extracted imapsync_1.xxx you see 6 files and 2 directories:
+ In the folder extracted and called imapsync_2.200, you see 7 files
+ and 2 directories.  Those files and directories may be presented in a
+ different order than the following, the order is not important
+ anyway. There are only two important files to get your mailbox
+ transfer job started in a few minutes, the first two files of the
+ following list:
 
-* README_Windows.txt     is the current file you are reading.
-* README.txt             is the imapsync general document.
-* FAQ.d/*                FAQs are a good read when something goes wrong.
-* imapsync_example.bat   is a simple batch file example you will copy and edit.
-* sync_loop_windows.bat  is a batch file example for syncing many accounts.
-* file.txt               is an input file example for syncing many accounts.
-* imapsync.exe           is the imapsync binary. You don't have to run it directly.
-* Cook/                  is the directory to build imapsync.exe from its source, 
-                         for the B) way and expert users.
+ * README_Windows.txt     is the current file you are reading.
+ * imapsync_example.bat   is a simple batch file example that you will copy and edit.
 
-You can copy or rename the file imapsync_example.bat as you wish,
-as long as its extension remains ".bat". 
-On Windows systems .bat extension means "I'm a batch script".
-Same thing for sync_loop_windows.bat.
-The batch scripts have to stay in the same directory than 
-imapsync.exe because of the way they call imapsync.exe, 
-they use ".\imapsync.exe", so let them be in the same directory.
-Or change the path .\ to whatever you want if you understand what 
-you're doing (you have to use a pathname from the script point of view).
+ * README.txt             is the imapsync general document.
+ * FAQ.d/*                FAQs are a good read when something goes wrong.
+ * sync_loop_windows.bat  is a batch file example for syncing many accounts.
+ * file.txt               is an input file example for syncing many accounts.
+ * imapsync.exe           is the imapsync 64bit binary. You don't have to run it directly.
+ * imapsync_32bit.exe     is the imapsync 32bit binary. You don't have to run it directly.
+ * Cook/                  is the directory to build imapsync.exe from its source, 
+                          for the B) way and expert users.
 
-For the rest of this documentation I assume you copied
-imapsync_example.bat to a file named imapsync_stuff.bat
+ You can copy or rename the file imapsync_example.bat as you wish,
+ as long as its extension remains ".bat", for example, mysync.bat
+
+ On Windows systems, a file name ending with a .bat extension means
+ "I'm a batch script".  A batch script is a file containing commands,
+ it's a program.  Don't be afraid, a program can be very simple to
+ modify and I hope imapsync_example.bat is one of them.
+
+ The batch scripts have to stay in the same directory as imapsync.exe
+ because of the way they call imapsync.exe.  They use the string
+ ".\imapsync.exe", so let them be in the same directory.
+
+ You can change the path .\ to whatever you want if you understand
+ what you are doing (you have to use a pathname from the script point
+ of view).
+
+ For the rest of this documentation, I assume you copied
+ imapsync_example.bat to a file named mysync.bat
+
+ If you don't know how to copy and rename a file then use
+ imapsync_example.bat itself, it's ok. The original file is still in
+ the zip file in case you want to restart from scratch.
 
 A.4) Edit the batch file
 
-Edit imapsync_stuff.bat and change the values with yours.
-In order to edit it you have do a right click on it and select "modify"
-in the list presented in the small window menu.
-Notepad or Notepadd++ are very good editors to modify it.
-Office Word or any powerful text processor are not good for that job, 
-don't use them!
+ This section describes how to edit the file mysync.bat and change 
+ it by replacing example values with your values.
+ 
+ To edit mysync.bat, you have to right-click on it and select "modify"
+ in the list presented in the small window menu.
 
-Files FAQ.txt and FAQ.d/* contain many tips and special options sometimes
-needed by specific imap server softwares like Exchange or Gmail.
+ Notepad or Notepadd++ are very good editor candidates to modify the
+ script.  Notepad is already installed on any Windows system,
+ Notepadd++ is not usually installed but if you have it, then use it.
 
+ Office Word or any powerful text processor is NOT good for that job.
+ Text processors transform files in a special format that is wrong to
+ make them stay a good batch file, so don't use them!
+
+ The documents FAQ.txt and FAQ.d/* contain many tips. They describe
+ some special options that are sometimes needed by specific imap
+ software servers like Exchange, Office365, or Gmail.
+
+ Those documents are also available online at
+ https://imapsync.lamiral.info/FAQ.d/
+ You don't have to look into them unless you encounter problems.
 
 A.5) Run the batch file
 
-To run imapsync with your values just double-click on 
-the batch file imapsync_stuff.bat
+ To run imapsync with your values just double-click on the batch file
+ mysync.bat
 
-There is no need to have administrator privileges to run imapsync.
+ There is no need to have administrator privileges to run imapsync.
 
-A.6) Loop on A.4) A.5) edit, run, edit, run etc.
+ The run happens in a DOS window; usually, this window is black.
 
-Loop the process of editing and running imapsync until
-you solve all issues and all values suit your needs.
+ If imapsync.exe returns immediately with the ERRORLEVEL -1, it may be
+ because you have a Group Policy in place to prevent the execution of
+ programs in the %TEMP% directory. Temporarily remove this restriction
+ and imapsync will work as expected. Thanks to Walter H. for this
+ input!
 
-A.7) Look the sync running. You can abort it at any time with a 
- quick double ctrl-c, hit ctrl-c twice within one second.
- (a single ctrl-c will reconnect to both imap servers).
- You can also abort the sync by closing the DOS window.
+ Technically speaking, imapsync.exe is an embedded Perl script with
+ also the Perl interpreter and many Perl modules, all glued together
+ in an archive auto-extracted at run time. So it needs write-access to
+ the temporary directory. The temporary directory name depends on the
+ user but its value is in the variable %TEMP%. You can have the value
+ by running the command ECHO %TEMP% in a DOS window. You can have a
+ DOS window by launching the command cmd.exe
 
-A.8) When the sync is finished you can find the whole log file 
-of the output in the folder named "LOG_imapsync", 
-the logfile name is based on the launching date, 
-hour, minute, second, milliseconds and the user2 parameter. 
+A.6) Look at the sync running. 
 
-There is one logfile per run.
+ You can abort the sync at any time with a quick double ctrl-c, hit
+ ctrl-c twice within one second.
+ 
+ A single ctrl-c will reconnect to both imap servers.
+ 
+ You can also simply abort the sync by closing the DOS window, using
+ the cross situated at the up-right corner.
 
-The logfile name is printed at the end of the imapsync run.
-If you do not want logging to a file then use option --nolog
+ What you see in this DOS terminal is also put in a logfile located
+ in the subdirectory LOG_imapsync/
+
+A.7) Control what happened.
+
+ When the sync is finished you can find the whole log file of the
+ output in the folder named "LOG_imapsync/".
+ 
+ The logfile name is based on the launching date, hour, minute,
+ second, milliseconds, plus the user1 and user2 parameters. 
+
+ For example, a file name can be
+ LOG_imapsync\2019_11_29_14_49_36_514_tata_titi.txt
+
+ There is one log file created for each run.  The log file name is
+ printed both at the beginning and the end of the imapsync run.
+
+ IMPORTANT: When there is a problem, the problem is very often
+ described at the end of the log file. It means you don't have to read
+ all this bloody ununderstandable verbose logfile, just read the end
+ first.
 
 
-B) Hard way. It is the hard way because it installs all software
-   dependencies. This is the way for modifying imapsync.exe if needed.
+A.8) Loop on A.4 through A.7
+
+ * A.4) edit the batch file
+ * A.5) run the batch file
+ * A.6) look at the run and the log file
+ * A.7) control what happened.
+
+ Loop on the process of editing, running and controlling imapsync
+ until you solve all issues and the sync is over.
+
+A good sign that the sync went very well is when the nearly last lines are like:
+
+" The sync looks good, all 123456 identified messages in host1 are on host2.
+" There is no unidentified message
+" Detected 0 errors
+
+Congratulations!
+
+
+------------
+B) Hard way
+------------
+
+It is the hard way because it installs all software dependencies.
+This is the way for modifying imapsync.exe if needed.
 
 B.1) Install Perl if it isn't already installed.
   Strawberry Perl is a very good candidate
   http://strawberryperl.com/
-  I use 5.26.0.1 (31 may 2017) but previous and later releases 
-  should work (5.18 and 5.20 do) as well.
+  I use 5.32.1.1 (released 2021-01-24) but previous and later releases 
+  should work as well (Perl 5.18 to 5.30 do).
 
 B.2) Go into the Cook/ directory
-B.3) Double-clic build_exe.bat 
+B.3) Double-click build_exe.bat 
 
 It should create a binary imapsync.exe in the current Cook/ directory.
 
 B.4) Move imapsync.exe in the upper directory and follow instructions
      from A.3) to A.8)
-

S/bc-payment.html

@@ -1,21 +0,0 @@
-
-<!-- $Id: bc-payment.html,v 1.12 2018/02/07 01:03:31 gilles Exp gilles $ -->
-
-
-<a 
-class="coinbase-button" 
-data-code="6a986b9f1386effacf76b350070c93ab" 
-href="https://www.coinbase.com/checkouts/6a986b9f1386effacf76b350070c93ab"
-><img  
-        alt="Pay with Bitcoins"
-        src="S/images/bitcoin_buy_now_large.png"
-/></a>
-
-
-<!-- 
-<script 
-src="https://www.coinbase.com/assets/button.js" 
->
-</script>
-
--->

S/conversion_data.js

@@ -0,0 +1,13 @@
+
+
+/* Google Code for Achat imapsync Conversion Page */
+
+var google_conversion_id = 992916412;
+var google_conversion_language = "en";
+var google_conversion_format = "2";
+var google_conversion_color = "ffffff";
+var google_conversion_label = "hVVWCKzApQIQvOe62QM";
+var google_conversion_value = "USD 1.00";
+var google_conversion_currency = "USD";
+var google_remarketing_only = false;
+

S/donate.shtml

@@ -29,9 +29,19 @@
 
 
 <body>
-<h1>imapsync donation</h1>
+<h1>Imapsync donation</h1>
+
+
+<p>Help the imapsync author, Gilles LAMIRAL, to maintain imapsync and its online services!</p>
+
+<div  id="right-tronche">
+<img  width="300"  height="300"  src="S/images/gilles_lamiral_400x400.jpeg"  alt="Gilles LAMIRAL" />
+</div>
+
+
+
+<h2>Donate via Paypal</h2>
 
-<p>Help the author to maintain imapsync and its online services:</p>
 <form action="https://www.paypal.com/cgi-bin/webscr" method="post">
 <p>
 <input type="hidden" name="cmd" value="_s-xclick"/>
@@ -43,11 +53,35 @@ value="-----BEGIN PKCS7-----MIIHRwYJKoZIhvcNAQcEoIIHODCCBzQCAQExggEwMIIBLAIBADCB
 </p>
 </form>
 
-<p>Thanks in advance!</p>
+<h2>Donate via Github</h2>
+
+<p> 
+<a href="https://github.com/sponsors/gilleslamiral">Become a Github sponsor to Gilles LAMIRAL</a>
+</p>
+
+<h2>Donate with Crypto</h2>
+<p>
+This donate button will ask your email address and your name but don't hesitate
+to give fake ones, like "Smith" and smith@example.com if you prefer to be anonymous.
+</p>
+<div>
+  <a class="donate-with-crypto"
+     href="https://commerce.coinbase.com/checkout/d757d82d-5f54-428e-9260-90c8b032d7ce">
+    <span>Donate with Crypto</span>
+  </a>
+  <script type="text/javascript" src="https://commerce.coinbase.com/v1/checkout.js?version=201807">
+  </script>
+</div>
+
+
+<p>Thanks!<br />
+And remember, <b>no money donation is ok!</b><br />
+Kind words are always sweets to me</p>
 
 <hr />
 <p>
 
+
 <a  href="http://validator.w3.org/check?uri=referer">
   <img  width="88"  height="31"  alt="Valid XHTML 1.0 Strict"  src="../S/images/valid-xhtml10" />
 </a>
@@ -64,7 +98,7 @@ value="-----BEGIN PKCS7-----MIIHRwYJKoZIhvcNAQcEoIIHODCCBzQCAQExggEwMIIBLAIBADCB
 <!--#config timefmt="%D" -->
 <!--#config timefmt="%A %B %d, %Y" -->
 <b>This document last modified on <!--#echo var="LAST_MODIFIED" --></b>
-($Id: paypal.shtml,v 1.13 2018/08/21 12:07:50 gilles Exp gilles $)
+($Id: donate.shtml,v 1.20 2021/01/01 22:38:02 gilles Exp gilles $)
 </p>
 
 </body>

S/external.shtml

@@ -26,22 +26,27 @@
 
 <body>
 
-<h1>Similar software <a id="similar" href="../#TOP"><small>(back to menu)</small></a>
+<h1>Imapsync similar software tools <a id="similar" href="../#TOP"><small>(back to menu)</small></a>
 </h1>
 
+<p>Here is a list of Imapsync alternative tools, list verified on Friday July 1, 2021:
+</p>
 
-<ul>
+<ol>
 <li> <b>imapsync</b>:    <a href="https://github.com/imapsync/imapsync">https://github.com/imapsync/imapsync</a> (this is an imapsync copy, sometimes delayed, with --noreleasecheck by default since release 1.592, 2014/05/22)</li>
-<li> imap_tools:         <a href="https://web.archive.org/web/20161228145952/http://www.athensfbc.com/imap_tools/">http://www.athensfbc.com/imap_tools/</a>.
+<li> imap_tools:         <a href="https://web.archive.org/web/20161228145952/http://www.athensfbc.com/imap_tools/">https://web.archive.org/web/20161228145952/http://www.athensfbc.com/imap_tools/</a>.
      The imap_tools code is now at <a href="https://github.com/andrewnimmo/rick-sanders-imap-tools">https://github.com/andrewnimmo/rick-sanders-imap-tools</a></li>
 <li> imaputils:          <a href="https://github.com/mtsatsenko/imaputils">https://github.com/mtsatsenko/imaputils</a> (very old imap_tools fork)</li>
-<li> Doveadm-Sync:       <a href="https://wiki2.dovecot.org/Tools/Doveadm/Sync">https://wiki2.dovecot.org/Tools/Doveadm/Sync</a> ( Dovecot sync tool )</li>
+<li> Doveadm-Sync:       <a href="https://wiki2.dovecot.org/Tools/Doveadm/Sync/">https://wiki2.dovecot.org/Tools/Doveadm/Sync</a> ( Dovecot sync tool )</li>
 <li> davmail:            <a href="http://davmail.sourceforge.net/">http://davmail.sourceforge.net/</a></li>
 <li> <b>offlineimap</b>: <a href="http://offlineimap.org/">http://offlineimap.org/</a></li>
+
+<li> fdm:                <a href="https://github.com/nicm/fdm">https://github.com/nicm/fdm</a></li>
+
 <li> <b>mbsync</b>:      <a href="http://isync.sourceforge.net/">http://isync.sourceforge.net/</a></li>
 <li> mailsync:           <a href="http://mailsync.sourceforge.net/">http://mailsync.sourceforge.net/</a></li>
-<li> mailutil:           <a href="http://www.washington.edu/imap/">http://www.washington.edu/imap/</a>
-                         part of the UW IMAP tookit.</li>
+<li> mailutil:           <a href="https://web.archive.org/web/20191028114408/http://www.washington.edu/imap/">https://www.washington.edu/imap/</a>
+                         part of the UW IMAP toolkit. (well, seems abandoned now)</li>
 <li> imaprepl:           <a href="https://bl0rg.net/software/">https://bl0rg.net/software/</a>
                          <a href="http://freecode.com/projects/imap-repl/">http://freecode.com/projects/imap-repl/</a></li>
 <li> imapcopy (Pascal):  <a href="http://www.ardiehl.de/imapcopy/">http://www.ardiehl.de/imapcopy/</a></li>
@@ -52,38 +57,106 @@
 <li> imapmigrate:        <a href="http://sourceforge.net/projects/cyrus-utils/">http://sourceforge.net/projects/cyrus-utils/</a></li>
 <li> larch:              <a href="https://github.com/rgrove/larch">https://github.com/rgrove/larch</a> (derived from wonko_imapsync, good at Gmail)</li>
 <li> wonko_imapsync:     <a href="http://web.archive.org/web/20130807173030/http://wonko.com/post/ruby_script_to_sync_email_from_any_imap_server_to_gmail">http://wonko.com/article/554</a> (superseded by larch)</li>
-<li> pop2imap:           <a href="http://www.linux-france.org/prj/pop2imap/">http://www.linux-france.org/prj/pop2imap/</a></li>
+<li> pop2imap:           <a href="https://pop2imap.lamiral.info/">https://pop2imap.lamiral.info/</a> (I wrote that too)</li>
 <li> exchange-away:      <a href="http://exchange-away.sourceforge.net/">http://exchange-away.sourceforge.net/</a></li>
-<li> SyncBackPro         <a href="http://www.2brightsparks.com/syncback/sbpro.html">http://www.2brightsparks.com/syncback/sbpro.html</a></li>
+<li> SyncBackPro:        <a href="http://www.2brightsparks.com/syncback/sbpro.html">http://www.2brightsparks.com/syncback/sbpro.html</a></li>
 
-<li> ImapSyncClient      <a href="https://github.com/ridaamirini/ImapSyncClient">https://github.com/ridaamirini/ImapSyncClient</a></li>
-<li> MailStore           <a href="https://www.mailstore.com/en/products/mailstore-home/">https://www.mailstore.com/en/products/mailstore-home/</a></li>
-<li> mnIMAPSync          <a href="https://github.com/manusa/mnIMAPSync">https://github.com/manusa/mnIMAPSync</a></li>
-<li> imap-upload          <a href="http://imap-upload.sourceforge.net/">http://imap-upload.sourceforge.net/</a></li>
+<li> ImapSyncClient:     <a href="https://github.com/ridaamirini/ImapSyncClient">https://github.com/ridaamirini/ImapSyncClient</a></li>
+<li> MailStore:          <a href="https://www.mailstore.com/en/products/mailstore-home/">https://www.mailstore.com/en/products/mailstore-home/</a></li>
+<li> mnIMAPSync:         <a href="https://github.com/manusa/mnIMAPSync">https://github.com/manusa/mnIMAPSync</a></li>
+<li> imap-upload:        <a href="http://imap-upload.sourceforge.net/">http://imap-upload.sourceforge.net/</a> 
+                         (A tool to upload a local mbox file to IMAP4 server)</li>
+<li> imapbackup:         <a href="https://github.com/rcarmo/imapbackup">https://github.com/rcarmo/imapbackup</a> 
+                         (A Python script to incremental backups of IMAP mailboxes)</li>
 
-</ul>
+<li> imap-backup:         <a href="https://github.com/joeyates/imap-backup">https://github.com/joeyates/imap-backup</a> 
+                         (A Ruby script to backup GMail (or other IMAP) accounts to disk)</li>
+
+
+<li> BitRecover email-backup <b>99 USD, 299 USD</b> <a href="https://www.bitrecover.com/email-backup/">https://www.bitrecover.com/email-backup/</a>.</li>
+
+<li> ImportExportTools:  <a href="https://addons.thunderbird.net/en-us/thunderbird/addon/importexporttools/">https://addons.thunderbird.net/en-us/thunderbird/addon/importexporttools/</a> 
+                        ImportExportTools for Mozilla Thunderbird by Paolo Kaosmos. ImportExportTools does not do IMAP.
+</li>
+
+<li> rximapmail:     <a href="https://sourceforge.net/projects/rximapmail/">https://sourceforge.net/projects/rximapmail/</a></li>
+
+<li> CodeTwo: <a href="https://www.codetwo.com/">https://www.codetwo.com/</a> but CodeTwo does
+<a href="https://www.codetwo.com/userguide/office-365-migration/imap-source.htm"> imap source to Office365 </a> only.</li>
+
+
+</ol>
 
 <h1>External online IMAP migration services <a id="ONLINE_OTHERS" href="../#TOP"><small>(back to menu)</small></a>
 </h1>
 
-<p>I have  no share in these free or paid services. 
-I don't think they use Imapsync. 
-Prices are given par mailbox and may be outdated (December 2011).</p>
+<p>These services are similar to the 
+<a  href="https://imapsync.lamiral.info/X/" title="Free online service under 3 GB">imapsync <b>online service</b></a>
+I call /X. Some are an exact copy of /X, some are not, some use Imapsync, some don't.<br/>
 
-<ul>
-<li> French Ovh imapcopy <b>0 EUR</b>:   <a href="https://mail.ovh.net/fr/imapcopy/">https://mail.ovh.net/fr/imapcopy/</a></li>
+I have  no share in these free or paid services. 
+Prices are given par mailbox and may be outdated (Last checked on Saturday 19 February 2022).
+</p>
+
+<h2>Imapsync based</h2>
+
+<!--
+From https://lamiral.info/dokuwiki/doku.php?id=imapsync&#online_ui_x_copies
+
+
+#  * 1.810 https://tools.controlpanel.si/imapsync/ CGI https://tools.controlpanel.si/cgi-bin/imapsync
+#  * 1.882 https://tools.intertune.io/imapsync/X/  CGI https://tools.intertune.io/cgi-bin/imapsync
+#  * 1.882 https://imapsync.whc.ca/                CGI https://imapsync.whc.ca/cgi-bin/imapsync
+#  * 1.925 https://imapsync.boomhost.com/          CGI https://imapsync.boomhost.com/cgi-bin/imapsync 
+#  * 1.977 https://imapcopy.webhosting4u.gr/       CGI https://imapcopy.webhosting4u.gr/cgi-bin/imapsync
+#  * 1.991 https://imapsync.nl/                    CGI https://imapsync.nl/cgi-bin/imapsync
+#  * 1.998 https://mailsync.timetakernet.info/     CGI https://mailsync.timetakernet.info/cgi-bin/imapsync
+#  * 1.998 https://imapsync.keurigonline.nl/       CGI https://imapsync.keurigonline.nl/cgi-bin/imapsync
+#
+#  * ?.??? https://app.migrationwizard.co.uk/      CGI 404, deep into https://app.migrationwizard.co.uk/sync.php
+-->
+
+<p>1.945 and upper use --useheader "Message-Id" as default in CGI mode and 
+this is a better choice to avoid duplicates with some imap servers.
+</p>
+
+<ol>
+<li> NameHero               <b>0 USD</b>: <a href="https://imapsync.net/">https://imapsync.net/</a> (imapsync 2.140)</li>
+<li> Slovenia Control Panel <b>0 USD</b>: <a href="https://tools.controlpanel.si/imapsync/">https://tools.controlpanel.si/imapsync/</a> (imapsync 1.810)</li>
+<li> Intertune Cloud        <b>0 USD</b>: <a href="https://tools.intertune.io/imapsync/X/" >https://tools.intertune.io/imapsync/X/</a> (imapsync 1.882)</li>
+<li> Web Hosting Canada     <b>0 USD</b>: <a href="https://imapsync.whc.ca/"               >https://imapsync.whc.ca/</a> (imapsync 1.882)</li>
+<li> BoomHost               <b>0 USD</b>: <a href="https://imapsync.boomhost.com/"         >https://imapsync.boomhost.com/</a> (imapsync 1.925)</li>
+<li> WebHosting|4u (greek)  <b>0 USD</b>: <a href="https://imapcopy.webhosting4u.gr/"      >https://imapcopy.webhosting4u.gr/</a> (imapsync 2.178)</li>
+<li> Vimexx                 <b>0 USD</b>: <a href="https://imapsync.nl/"                   >https://imapsync.nl/</a> (imapsync 1.991)</li>
+<li> KeurigOnline           <b>0 USD</b>: <a href="https://imapsync.keurigonline.nl/"      >https://imapsync.keurigonline.nl/</a> (imapsync 1.998)</li>
+</ol>
+
+<h2>Not Imapsync based</h2>
+<ol>
+<li> Ovh migration <b>0 EUR</b>:         <a href="https://omm.ovh.net/Migration/Create">https://omm.ovh.net/Migration/Create</a></li>
 <li> Rackspace migration <b>0 USD</b>:   <a href="http://www.rackspace.com/email-hosting/migrations">http://www.rackspace.com/email-hosting/migrations</a></li>
-<li> Movemymail one free, 5 USD after:   <a href="https://movemymail.net">https://movemymail.net/</a></li>
-<li> TransferMyEmail <b>?? USD</b>:      <a href="https://transfermyemail.ca/">https://transfermyemail.ca/</a></li>
-<li> Migrationwiz 12 USD:                <a href="https://www.bittitan.com/products/migrationwiz/">https://www.bittitan.com/products/migrationwiz/</a> 
-    ( See this remarkable comparison     <a href="https://blog.bittitan.com/imapsync-vs-migrationwiz/">Imapsync vs Migrationwiz</a>! )</li>
-<li> Audriga   9.99 EUR:                 <a href="https://www.groupware-migration.com/">https://www.groupware-migration.com/</a></li>
-<li> Yippiemove 14.95 USD:               <a href="http://www.yippiemove.com">http://www.yippiemove.com/</a> 
-     ( See <a href="https://www.yippiemove.com/help/faq_generic.html#imapsync">Yippiemove vs ImapSync</a> )</li>
+<li> Movemymail one free, <a href="https://movemymail.net/faq.php">5 USD</a> after: <a href="https://movemymail.net">https://movemymail.net/</a></li>
+<li> TransferMyEmail <a href="https://transfermyemail.ca/shop/">1.50 USD</a> for a day pass: <a href="https://transfermyemail.ca/">https://transfermyemail.ca/</a></li>
+<li> Migrationwiz <a href="https://www.bittitan.com/pricing/">12 USD</a>: <a href="https://www.bittitan.com/products/migrationwiz/">https://www.bittitan.com/products/migrationwiz/</a> </li>
+<li> Audriga <a href="https://www.audriga.com/en/Self-service_migration/Pricing">11.90 EUR</a>: <a href="https://www.audriga.com/">https://www.audriga.com/</a></li>
+<li> <del>Yippiemove 14.95 USD</del> (discontinued on April 2019):    <a href="http://www.yippiemove.com">http://www.yippiemove.com/</a> 
+     ( See <a href="https://web.archive.org/web/20161027124328/https://www.yippiemove.com/help/faq_generic.html#imapsync">Yippiemove vs ImapSync</a> )</li>
 <li> ShuttleCloud one free, 19.95 USD after <a href="https://www.shuttlecloud.com/">https://www.shuttlecloud.com/</a></li>
+<li> Cloudasta ??? USD:                  <a href="https://cloudasta.com/">https://cloudasta.com/</a></li>
+</ol>
 
-</ul>
+<h2>Imap to files services</h2>
 
+<p>
+This services allow you to backup/archive your imap email account to a zip file.
+The most common and useful format to restore an imap account is the <b>Maildir</b> format.
+</p>
+
+<ol>
+<li>PowerMail Backup (French language)
+    <a href="https://www.powermail.fr/sauvegarder-une-boite-mail/">https://www.powermail.fr/sauvegarder-une-boite-mail/</a>.
+</li>
+</ol>
 
 <hr/>
 <div>
@@ -111,7 +184,7 @@ alt="Viewable With Any Browser" />
 <!--#config timefmt="%D" -->
 <!--#config timefmt="%A %B %d, %Y" -->
 <b>This document last modified on <!--#echo var="LAST_MODIFIED" --></b>
-($Id: external.shtml,v 1.28 2019/02/17 14:42:26 gilles Exp gilles $)<br/>
+($Id: external.shtml,v 1.61 2022/02/19 20:26:25 gilles Exp gilles $)<br/>
 <a href="#TOP">Top of the page</a>
 </p>