Maildir to IMAP

Discussion in 'IMAP Tools' started by popowich, Oct 27, 2013.

  1. popowich

    popowich EQ Forum Admin Staff Member

    Joined:
    Aug 12, 2008
    Messages:
    8,983
    Likes Received:
    120
    maildir_to_imap.pl

    maildir_to_imap is used to copy the messages in a maildir to a user's IMAP mailbox. maildir_to_imap is executed like this:

    ./maildir_to_imap.pl -i <user list> -D <imapserver[: port]>

    The user list is a file with one or more entries containing the location of the user's maildir and his IMAP username and password.

    For example:

    /mhub4/maildirs/rwilson@abc.net,rich.wilson,welcome
    /mhub4/maildirs/jane.eyre@abc.net,jane.eyre,mypass

    See usage() for a list of arguments

    If you have a question about using this script please reply.

    If you would like to download the script please see How to get the IMAP Tools.
     


  2. Jason Hunter

    Jason Hunter New Email

    Joined:
    Sep 19, 2015
    Messages:
    8
    Likes Received:
    0
    I'm really struggling to figure this one out.

    I have a directory called hukarz. Inside it I have directories:

    hukarz/foo
    hukarz/bar
    hukarz/baz

    Each directory is most a mailinglist, like List-ID: foo@foo.foo

    Well, anyways, I've admitted defeat and I've now signed up for a hosted IMAP account, so I want to upload each directory into the corresponding IMAP directory.

    I've tried running:

    maildir_to_imap.pl -d -M mob/ -A my-email-address:my-password -D email.emailarray.com

    ..where mob is a directory inside hukarz.

    The output is:

    Can't open user list : No such file or directory at /home/b0ef/Nedlastinger/imap-tools/maildir_to_imap.pl line 565.

    I don't know what a user list is. Is that specified anywhere?
     

  3. Jason Hunter

    Jason Hunter New Email

    Joined:
    Sep 19, 2015
    Messages:
    8
    Likes Received:
    0
    ok, I think I've got it; let me try.
     
  4. Jason Hunter

    Jason Hunter New Email

    Joined:
    Sep 19, 2015
    Messages:
    8
    Likes Received:
    0
    ok, I'm closer.

    It seems this utility requires the actual directory to have a "." in front of it, like .foo and .bar
    How can I say that I don't have dots in front of the directory names?

    Also, it seems to require a subdirectory called "cur".

    I know this is the proper way for a Maildir, but my email application, gnus, didn't use that. I only have a directory for each group, like hukarz/foo and then messages directly in there.
     
  5. rfs9999

    rfs9999 IMAP Tools

    Joined:
    Oct 27, 2013
    Messages:
    114
    Likes Received:
    6
    I think you are using the wrong tool for the job.

    Like you said, maildir_to_imap is designed for importing a maildir-formatted directory structure. While you could create the proper structure --- it's pretty easy if you understand how maildir works (Maildir - Wikipedia, the free encyclopedia --- it would be easier to use a different tool.

    Try dumptoIMAP.pl. It was designed to import messages into IMAP from a set of directories containing message files. The name of the directory is used for the IMAP folder name. A quick summary is below but see IMAP Tools for the details:

    dumptoIMAP.pl
    -i host/user/password
    -L dumptoIMAP.log
    -D <full path>/hukarz

    -D gives the path to the subdirectories containing the messages you want to import. In your case "-D <full path>/hukarz". dumptoIMAP.pl will read the messages in each subdirectory ("foo", "bar", and "baz"). An IMAP folder will be created for each one and the messages will be uploaded into it.

    -Rick
     
    popowich likes this.
  6. Jason Hunter

    Jason Hunter New Email

    Joined:
    Sep 19, 2015
    Messages:
    8
    Likes Received:
    0
    Indeed; that's just great;) Thanks.

    I see, however, that from my early years, my split filters were not that great, so I see that I've got some messages that have ended up in the wrong directory, so I got some messages from a certain mailinglist inside the wrong directory.

    You think I should clean all this up before or after I've uploaded to the IMAP server?

    I'm also wondering, shouldn't all these groups (I got like 400) go under the INBOX directory? I see they end up outside the INBOX, ie. with no parent?

    Also, I see that I'm not automatically subscribed to the group, so that I need to go into the Polaris Control Panel to subscribe. Is this something that also could be done via this script?

    Other than that, thank you; I'm in safe hands now;)
     
  7. rfs9999

    rfs9999 IMAP Tools

    Joined:
    Oct 27, 2013
    Messages:
    114
    Likes Received:
    6
    Jason,

    By default dumptoIMAP puts the folders at the top level rather than under the INBOX. It does have latent support for putting them under a "root" mailbox but I need to activate it in the code. Send me your e-mail address (to rfs9999@earthlink.net) and I will send you the new version.

    >I see that I'm not automatically subscribed to the group, so that I need to go into the Polaris Control Panel to subscribe. Is this something that also could be done via this script?

    By group you mean the IMAP mailbox that is created for the subdirectory with the messages, eg "Foo"? I can add the IMAP SUBSCRIBE command to the mailboxes it creates but I don't know if that will solve the problem. Most e-mail client have their own 'subscribe' mechanism and whether the mailbox has been subscribed in IMAP doesn't affect whether the client shows the mailboxes or not. I'll add it to the script anyway.

    -Rick
     
  8. Jason Hunter

    Jason Hunter New Email

    Joined:
    Sep 19, 2015
    Messages:
    8
    Likes Received:
    0
    Great, I've sent you an email.

    Also, isn't it the proper way to put it under INBOX?

    I'm not that familiar with how I'm supposed to do it, but it seems natural to put it cleanly under INBOX?
     
  9. rfs9999

    rfs9999 IMAP Tools

    Joined:
    Oct 27, 2013
    Messages:
    114
    Likes Received:
    6
    I just sent the new version to you. Use -r INBOX to tell dumptoIMAP to put all imported folders under the INBOX.

    When importing folders into IMAP it doesn't make sense to put them automatically under the INBOX, at least not to me. dumptoIMAP is often used to take a dump of an IMAP account and load it into another IMAP server or to restore a backup copy. In that case you want to preserve the folder structure that it was dumped from. For example:

    Inbox
    Notes
    Old_Stuff
    Receipts
    Receipts/2014
    Receipts/2014/November
    Receipts/2014/December
    Receipts/ 2015
    Receipts/2015/January
    etc

    dumptoIMAP retains the folder structure when it loads the folders and messages into the destination IMAP server.

    Instead of...

    hukarz/foo
    hukarz/bar
    hukarz/baz

    ... you could have put them under the INBOX without needing the -r <root folder> by creating a directory structure like this:

    hukarz/inbox/foo
    hukarz/inbox/bar
    hukarz/inbox/baz

    dumptoIMAP -D hukarz would have created INBOX/foo, INBOX/bar, and INBOX/baz folders.

    -Rick
     
  10. Jason Hunter

    Jason Hunter New Email

    Joined:
    Sep 19, 2015
    Messages:
    8
    Likes Received:
    0
    ok, great; thanks.

    I do however see a few of these:

    100 messages copied
    200 messages copied
    unexpected APPEND response to
    unexpected APPEND response to
    300 messages copied
    400 messages copied
    unexpected APPEND response to
    unexpected APPEND response to
    500 messages copied
    600 messages copied
    700 messages copied
    800 messages copied
    900 messages copied
    1000 messages copied
    unexpected APPEND response to
    1100 messages copied
    1200 messages copied
    1300 messages copied
    1400 messages copied
    1500 messages copied
    unexpected APPEND response to
    unexpected APPEND response to
    1600 messages copied
    1700 messages copied
    unexpected APPEND response to
    1800 messages copied
    1900 messages copied
    unexpected APPEND response to
    2000 messages copied
    2100 messages copied
    2200 messages copied
    2300 messages copied
    2400 messages copied
    2500 messages copied

    I've looked in the log, but it's actually empty, so I don't know if I should care about these unexpected APPEND messages.
     
  11. rfs9999

    rfs9999 IMAP Tools

    Joined:
    Oct 27, 2013
    Messages:
    114
    Likes Received:
    6
    Each 'unexpected APPEND response' represents a message that was not loaded into an IMAP folder.

    It may be that the date in the header of some of the messages is not formatted correctly. That is the most common reason that an IMAP server rejects an APPEND command. Some IMAP servers are unforgiving about malformed dates (not in the proper RFC822 format).

    If you run in debug mode with -d and -I (that's a capital EYE) then the logfile will record the IMAP commands and responses from the server. That will show what's wrong with those APPENDS.

    I don't understand why the logfile would be empty.

    -Rick
     
  12. Jason Hunter

    Jason Hunter New Email

    Joined:
    Sep 19, 2015
    Messages:
    8
    Likes Received:
    0
    I seem to be having a new problem now.

    The scripts just quits, silently.

    I got a directory of about 20 directories which I want it to process, but then it just stops after it has takes a few of them. It also starts in the middle, at letter j* for some unknown reason.

    The log file doesn't show anything.

    How can I troubleshoot this?
     
  13. rfs9999

    rfs9999 IMAP Tools

    Joined:
    Oct 27, 2013
    Messages:
    114
    Likes Received:
    6
    Please run it with the -d and -I and send me the logfile.
     
  14. Jason Hunter

    Jason Hunter New Email

    Joined:
    Sep 19, 2015
    Messages:
    8
    Likes Received:
    0
    That creates really a big logfile;)

    I've created a video demonstrating the problem. Maybe it's enough.

    You have maybe an email I could send it to, as it contains my IMAP password?
     
  15. rfs9999

    rfs9999 IMAP Tools

    Joined:
    Oct 27, 2013
    Messages:
    114
    Likes Received:
    6
  16. rfs9999

    rfs9999 IMAP Tools

    Joined:
    Oct 27, 2013
    Messages:
    114
    Likes Received:
    6
    The video is blurry and really hard to view on my screen. How big is the logfile after being zipped or gzipped? You can FTP it to the following site www.athensfbc.com. Using "anonymous" to log in, cd to incoming, and then do a "put <file>".
     

Want to reply or ask your own question?

It only takes a minute to sign up (and it's free!). Click the orange sign up button to choose a username and then you can ask your own questions on the forum.
Loading...