| Request | MaildropHost -- feature request -- by Gaël Le Mignot |
| Posted on | Mar 22, 2005 5:19 am |
| Subscribe |
| Comment by Gaël Le Mignot on Mar 24, 2005 10:14 am | |
|
I don't have any simple test cases, sorry. I'll try to do some if I get some free time, but I'm a bit overbooked those time :/ I understand your point, and yes, patching at this right location is always better - but IMO, having an extra security net is never a bad idea, and cannot do harm. Anyway, thanks for the time you spent on this issue. |
|
|
|
| Resolve by Jens Vagelpohl on Mar 24, 2005 5:59 am | |
|
Hi Gaël, This is what I have done at this point: - I integrated those parts of your patch that allow setting the PID file and log file locations - I added unit tests to the package that try to exercise various transaction situations and the behavior of the TransactionalEmail class My own testing and these new unit tests have never uncovered a situation where duplicate emails get sent. I simply cannot reproduce the problem. MaildropHost is an integral part of such software as Maik Jablonski's MailBoxer and it is being used to run the German Zope user group lists - situations in which you have several hundred email messages at once. No one has ever received duplicates. I am still very interested in uncovering a potential problem, if there is one. However, at this point I cannot see a problem in MaildropHost and have to blame the application that uses it. As I pointed out in earlier replies, MaildropHost is the wrong place to work around an application or a user misbehaving. If you can replicate the problem in a simple setup that I can easily create here I'll be more than happy to help. Please open a new tracker issue at that point, I will close this one as "resolved" now. Thanks a lot for your great patches! jens |
|
|
|
| Comment by Jens Vagelpohl on Mar 22, 2005 10:08 am | |
|
> Zope retrys the full external method/product method if a ConflictError > happens in it, so if a method sends a mail and then do something in > the ZOBB, if ConflictError happens in this second operation, Zope will > retry the full method, and the mail will be sent a second time. I know that everything gets retried, but there should still only be one mail, *except* if someone manually messes around with the Zope transaction. If there are manually induced commits then the mail will be put into the spool. But it is not the job of the MaildropHost to guard against that. I'm currently trying to write some unit tests to see if I can reproduce the problem. I can't really say much about the code you sent because it involves other products where I have no idea what they do or how they do it. I would need a simple test case that runs in a vanilla Zope instance. > This patch can also protect against user errors, like clicking twice > or doing a reload/back when he shouldn't have. The application that uses the MaildropHost needs protection for that. It's not the MaildropHosts's job to guard against problems in applications that utilize it. So far I see no reason for incorporating that part of the patch. It either glosses over a real bug in MaildropHost (duplicate emails despite the transactional behavior) or it solves a problem that needs to be solved elsewhere. jens |
|
|
|
| Comment by Gaël Le Mignot on Mar 22, 2005 5:45 am | |
|
It happened from time to time on several of our sites (usually when sending a bunch of mails at once, or when the site is overloaded), but not in a reproductible way. For the tests, I faked a test case by adding a "raise ConflictError" after sending the mail in PloneGazette, by changing: recipients = theme.mailingInfos() errors1 = self.sendToRecipients(recipients) recipients = theme.getExtraRecipients() errors2 = self.sendToRecipients(recipients) into: recipients = theme.mailingInfos() errors1 = self.sendToRecipients(recipients) raise ConflictError recipients = theme.getExtraRecipients() errors2 = self.sendToRecipients(recipients) Zope retrys the full external method/product method if a ConflictError happens in it, so if a method sends a mail and then do something in the ZOBB, if ConflictError happens in this second operation, Zope will retry the full method, and the mail will be sent a second time. This patch can also protect against user errors, like clicking twice or doing a reload/back when he shouldn't have. |
|
|
|
| Comment by Jens Vagelpohl on Mar 22, 2005 5:31 am | |
|
> - an option to ignore duplicates (same mail for same target) if send=20= > within a given timeframe (like, on hour). This is a work-around for a=20= > problem with Zope: if a ConflictError happens just after a mail was=20 > sent, Zope will retry the whole process, and the mail will be sent=20 > again, resulting in some users receiving duplicate mails. With this=20 > option, MaildropHost will ignore the second (and further) mails. Hi Ga=EBl, There should never be a second email. That's the whole point of the=20 transaction-awareness: Only a successful ZODB transaction commit will=20 put a piece of mail in the spool. If you really do see more than one=20 mail in one ZODB transaction (retried or not) then this is a bug. Do=20 you by any chance have some reproducible test case? jens |
|
|
|
| Initial Request by Gaël Le Mignot on Mar 22, 2005 5:19 am | |
|
Thanks for including the first patch :) Here is a second patch from PilotSystems, with two changes: - in the same way that the previous one with the spool/var split, this patch allows users to specify explicit pid file and log file (/var/run/maildrophost.pid and /var/log/maildrophost.log for example). - an option to ignore duplicates (same mail for same target) if send within a given timeframe (like, on hour). This is a work-around for a problem with Zope: if a ConflictError happens just after a mail was sent, Zope will retry the whole process, and the mail will be sent again, resulting in some users receiving duplicate mails. With this option, MaildropHost will ignore the second (and further) mails. The patch is uploaded in my account on ZopeVillage: http://www.zopevillage.com/Members/gael/maildrophost-duplicate.patch/download |
