From e64b7d281ae61973976edfea4263fa3035e0de3c Mon Sep 17 00:00:00 2001 From: Lucian Mogosanu Date: Tue, 19 Mar 2019 19:41:50 +0200 Subject: [PATCH] posts: 089 --- posts/y05/089-feedparse-ii.markdown | 75 +++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+) create mode 100644 posts/y05/089-feedparse-ii.markdown diff --git a/posts/y05/089-feedparse-ii.markdown b/posts/y05/089-feedparse-ii.markdown new file mode 100644 index 0000000..ce56ed9 --- /dev/null +++ b/posts/y05/089-feedparse-ii.markdown @@ -0,0 +1,75 @@ +--- +postid: 089 +title: A few changes to Feedparse's HTTP requester (prelude to Feedbot) +date: March 19, 2019 +author: Lucian Mogoșanu +tags: tech, tmsr +--- + +This post introduces s-xml-feedparse-http-fixes: + +* the [V patch][s-xml-feedparse-http-fixes.vpatch]; and +* my [seal][s-xml-feedparse-http-fixes.vpatch.spyked.sig]. + +So what do these changes slash "HTTP fixes" consist of, more precisely? +Two pieces of code are being repaired -- one of the repairs being +required by Feedbot, while the other is more or less an opportunity for +making the code a tad clea[nr]er. The latter (simpler) one goes as +follows: + +~~~~ {.diff} ++(defvar *http-request-timeout* 60) + +... + +- (parse-feed-string (http-request-with-timeout url 60))) ++ (parse-feed-string (http-request-with-timeout url *http-request-timeout*))) +~~~~ + +In other words, we're calling `http-request-with-timeout` with an +operator-configurable timeout setting -- as it should have been in the +first place, but let's not go into too many details, lest I end up +burning myself and applying ice on the wound. + +The former (more important) change is along the lines of: + +~~~~ {.diff} ++(defun http-request-thread (url timeout) ++ (handler-case ++ (drakma:http-request url :connection-timeout timeout) ++ ;; Note: this will have to be revisited once we have a sane view of ++ ;; all the conditions returned by drakma. ++ (t (c) ++ (return-from http-request-thread (values nil c))))) + +... and a bit later: + + (let ((req-thread (sb-thread:make-thread +- #'(lambda (url timeout) +- (drakma:http-request url :connection-timeout timeout)) ++ #'http-request-thread +~~~~ + +This means that the thread we're creating now runs the code in +`http-request-thread` instead of an anonymous function; but more +importantly, that the call to [Drakma][drakma]'s `http-request` is now +guarded by a [handler-case][handler-case] that, long story short, acts +as a wildcard exception catcher. + +Feedbot benefits from this change by making sure that all HTTP requests +-- that, as we know from above, occur on a separate thread -- are +deterministic, i.e. the CL run-time doesn't end up throwing exceptions +at us if somehow a request times out or who knows what other condition +out of the operator's control. I know, this isn't the most fortunate of +implementation choices, but well... if I do the right thing(tm) and fix +Drakma at this very moment rather than later, and then that other thing, +and then the other one, then I might get to publish Feedbot in a decade +or so. Let's say that the debt is manageable -- for now, but not for +long. + +Up next: Feedbot, part the first. + +[s-xml-feedparse-http-fixes.vpatch]: http://lucian.mogosanu.ro/src/s-xml/v/patches/s-xml-feedparse-http-fixes.vpatch +[s-xml-feedparse-http-fixes.vpatch.spyked.sig]: http://lucian.mogosanu.ro/src/s-xml/v/seals/s-xml-feedparse-http-fixes.vpatch.spyked.sig +[drakma]: /posts/y05/087-feedparse.html#selection-300.0-300.1 +[handler-case]: https://archive.is/x7yfi -- 1.7.10.4