From: Lucian Mogosanu Date: Fri, 26 Jul 2019 12:14:40 +0000 (+0300) Subject: posts: 097 X-Git-Tag: v0.11~22 X-Git-Url: https://git.mogosanu.ro/?a=commitdiff_plain;h=05fd31db6ffc3938d4fba92a5eb9566300f3c98e;p=thetarpit.git posts: 097 --- diff --git a/drafts/000-hunchentoot-iii.markdown b/drafts/000-hunchentoot-iii.markdown deleted file mode 100644 index d4c61aa..0000000 --- a/drafts/000-hunchentoot-iii.markdown +++ /dev/null @@ -1,370 +0,0 @@ ---- -postid: 000 -title: Hunchentoot: further architectural notes; and usage examples -date: July 25, 2019 -author: Lucian Mogoșanu -tags: tech, tmsr ---- - -This post is part of a series on [Common Lisp WWWism][cl-www], more -specifically a continuation of [ongoing work][hunchentoot-i] to -[understand][hunchentoot-ii] the web server known as Hunchentoot and, -as a result, produce a signed genesis to be used by members of the -TMSR [WoT][wot]. - -In this episode we'll do another illustration of the Hunchentoot -architecture; and we'll have fun documenting a running web server -instance, thusly exploring for now a few of things that it can do[^1]. - -First, the architectural diagram, with bells, whistles and clickable -stuff: - - -
- - - -
- -No, really, I'm not kidding: if you have a browser that implements -SVG, clicking on the text should direct you to defgeneric pieces of -code[^2]. Anyway, the big squares are Hunchentoot components and, more -specifically, the name of their CL classes, while the small squares -found inside the big ones represent methods specializing on a given -class. The green boxes are user actionable or defineable methods, so -this is where you should start looking; while the arrows denote the "X -calls Y" relation, with the exception of the dashed green arrow, that -tells us that header-out is in fact a setf-able accessor used from -somewhere within the context of acceptor-dispatch-request (e.g. from a -request handler) to read and modify the header of a reply. - -Now from this airplane view, Hunchentoot's organization looks quite -digestible, which should give us a very good idea of how to start -using it. So let's take a look at that, shall we? - -Assuming we've loaded[^3] Hunchentoot into our CLtron of choice, we -can now create an acceptor instance and start it: - -~~~~ {.commonlisp} -> (defvar *myaccept* - (make-instance 'hunchentoot:acceptor :port 8052)) -> (hunchentoot:start *myaccept*) -~~~~ - -... and now what? Say, for now, that we want to serve a static site -- -I'm using The Tar Pit as my playground, but you can use whatever you -fancy. Looking at -[acceptor-dispatch-request][ht-acceptor-dispatch-request], we notice -that it calls [handle-static-file][ht-handle-static-file] with the -[document-root][ht-document-root] as an argument. So let's set that, -and additionally the error template directory, to our site: - -~~~~ {.commonlisp} -(setf (hunchentoot:acceptor-document-root *myaccept*) - "/home/spyked/thetarpit/site/" - (hunchentoot:acceptor-error-template-directory *myaccept*) - "/home/spyked/thetarpit/site/") -~~~~ - -and now `curl http://localhost:8052` should serve its contents. - -But let's say we want to go one step further and serve some content -(server-side) dynamically. The original Hunchentoot -[documentation][ht-docs] actually provides a neat minimal example, -which I'm going to steal, but not before explaining what we're going -to do here. - -Besides serving files off the disk, a web server can do other useful -stuff, such as, in Apache's case, sending the file to a preprocessing -engine (PHP or whatever), or as we're going to show, executing some -other predefined action that depends on the request parameters (URL, -cookies, HTTP method, variables and so on). For now, let's say that we -want our server to respond to the URL "/yo" (where "/" is the site -root) with the plain-text message "Hey!". Furthermore, let's say that -we want to optionally parameterize requests to this URL by the -variable "name", in which case the response will include the name: for -example, if we do a GET request to "/yo?name=spyked", we want the -server to respond with "Hey, spyked!". - -We have a few possible ways of doing this. We could for example edit -the current implementation of -[acceptor-dispatch-request][ht-acceptor-dispatch-request], which is -also the ugliest possible approach. On the other hand, Hunchentoot is -built using Common Lisp's Object System mechanism (CLOS), which allows -us to subclass the acceptor to a user-defined class and specialize the -method above for our class. Let's try this out: - -~~~~ {.commonlisp} -(defclass myacceptor (hunchentoot:acceptor) - ()) -(change-class *myaccept* 'myacceptor) -~~~~ - -The change-class thing isn't something that we'd normally do, but if -you've been following along, you'll notice that this didn't break our -code, because well, Common Lisp is cool. Now for the dispatcher -method: - -~~~~ {.commonlisp} -(defmethod hunchentoot:acceptor-dispatch-request - ((acceptor myacceptor) request) - (cond - ((string= (hunchentoot:script-name request) "/yo") - (let ((name (cdr (assoc "name" (hunchentoot:get-parameters request) - :test #'string=)))) - (setf (hunchentoot:content-type*) "text/plain") - (format nil "Hey~@[, ~A~]!" name))) - (t (call-next-method)))) -~~~~ - -In human words: this is an implementation of acceptor-dispatch-request -specialized on myacceptor, that, upon encountering the URL -(script-name) "/yo", takes the value of the GET parameter known as -"name" and returns a response string (possibly containing this "name") -as plain text. Otherwise it transfers control to the "next most -specific method"[^4], implicitly passing to it the existing arguments. - -We could stop here, but we won't, as there's a short discussion to be -had, mainly related to the extensibility of our approach, i.e. what -happens when we add other custom URLs to this recipe? The naive result -will look ugly and will be a pain to maintain and debug; while the -more elaborate approach, involving putting every "/yo" into its own -function, will initially fill our implementation with cond/case -conditions, eventually leading to a more civilized dispatch mechanism, -in the form of a lookup table from URLs to handler functions. - -Well, it so happens that Hunchentoot already has an implementation for -this type of thing, going under the name of -[easy-acceptor][ht-easy-acceptor]. easy-acceptor defines a dispatch -table whose only dispatcher is (initially) the -[dispatch-easy-handlers][ht-dispatch-easy-handlers] function, which -looks up handlers for URLs in a global handler list, -\*easy-handler-alist\*. As things usually go with these -[domain-specific languages][cl-who-ii-fn4], most of the handler -maintenance work is piled up in the -[define-easy-handler][ht-define-easy-handler] macro. - -So, in order to illustrate this easy-stuff, first let's undo some of -our previous work and redo the very basics: - -~~~~ {.commonlisp} -(hunchentoot:stop *myaccept*) -(setq *myaccept* (make-instance 'hunchentoot:easy-acceptor - :port 8052 - :document-root "/home/spyked/thetarpit/site/" - :error-template-directory - "/home/spyked/thetarpit/site/")) -(hunchentoot:start *myaccept*) -~~~~ - -Notice how now we're instancing easy-acceptor. Now we can define an -equivalent "easy handler" for our previous "/yo" work: - -~~~~ {.commonlisp} -(hunchentoot:define-easy-handler (say-yo :uri "/yo") (name) - (setf (hunchentoot:content-type*) "text/plain") - (format nil "Hey~@[, ~A~]!" name)) -~~~~ - -which about sums up our exercise. Initially I had wanted to show an -example doing some fancy prefix/"smart" URL lookup à la -[MP-WP][mp-wp], but by now this post is so large that it can't be -eaten in one sitting. Alas, I will have to leave all my fancy examples -for another episode. Thus, until next time... - -[^1]: Contrary to popular beliefs and expectations, the things that - some particular X can do that are known to (some particular) me - are not to be confused with the total set of things that said X - can possibly do, nor with the set of things that it *can't* - do. Take for example X = your average pointeristic slash - buffer-overflowistic C barfola: you can identify some particular - uses for it, sure, but meanwhile the average douchebag will - exercise code paths that may make it usable for things you've - never imagined, such as stealing your keys, wiping your disk and - murdering your dog... and many other things, short of making you - some french fries, which is something that e.g. a web server can't - do. - - In other words, nobody gives a fuck about popular beliefs and - expectations; and by the time I publish a [signed][io] genesis for - this Hunchentoot thing -- good, bad, with or without warts or - however we have it -- I will be entirely able to say what it does - and doesn't do, which is exactly what I'm working on here and now. - - And now to be an asshole and leave this otherwise properly rounded - footnote hanging: what about, say, [usocket][usocket]? and then - what about SBCL or some other working CLtron? and what about - [Linux][btcbase-1923536] and [its userland][cuntoo]? This - unfortunately is the curse of our postmodern times: our ability to - run computing machines rests, for the time being, upon the promise - of some [shitheads][btcbase-1918973]. - -[^2]: I don't write HTML and CSS for a living, so I might as well use - this footnote to document the pain required to generate this, for - later reference. - - Specifying the diagram in GraphViz is fairly straightforward: one - simply has to list the clusters, the nodes and the edges within - them in a text file -- see for example the final [.dot - file][hunchentootarch-dot] used for generating the illustration - above. Adding links and colours and all that is also easy, as - previously shown. The problem, however, with this GraphViz thing - is that graph generation involves an automated step, i.e. node - layout generation and edge routing, that can easily prove to be a - pain in the ass for the user: not only do I want this diagram - generated, but I also want it to be arranged like *so*, and not - like *that*, because I want the viewer to be able to look at the - components of the graph in some particular order. - - To add insult to injury, this automated step is almost entirely - opaque to the user: in order to have that square near that one, I - need to frantically shuffle nodes and edges about until I find the - magic ordering that generates something close to what I want -- - that is, the relationship between said ordering and the output is - purely coincidental, and I'm stuck guessing based on the vague - hints found in the spec. Anyway, this is the best diagram layout - we've got here at The Tar Pit, sorry... do make sure to write in - if I'm in the wrong. - - Now that I have a representation, I need to embed it in the blog - post. One would expect that's also straightforward, wouldn't he? - Well, no! You see, I got the idea that placing clickable links in - generated SVG files is cool, only this doesn't work in the - slightest when inserting the "<img>" tag, because completely - counter-intuitively for a SVG, the browser displays *an image*, - not a DOM sub-tree. So then I look at how Phf did it with his - [patch viewer][btcbase-patches], and it looks like he's inserting - a HTML image-map in the HTML document, which kinda beats the - purpose of having links in the SVG in the first place. I really, - *really* don't want to copy-paste the whole diagram into the post, - so what the fuck am I gonna do, use <object> tags?! - - So if by now you were curious enough to look at the page source, - you'll notice that what I did was to insert an inline <svg> - that then imports the content of my .svg file using the - [<use>][svg-use] tag, which works exactly the way I want - it. And no, you won't find this anywhere on Google either, because - Google [doesn't fucking work][btcbase-1922361]. - - To sum this up: IMHO the result looks pretty cool, with the - mention that I'm most likely going to write the SVG diagram "by - hand" next time I'm doing anything non-trivial. At least then no - magic tool will lie to me that it saves hours of my work, when it - instead adds to it. - -[^3]: Since I'm trying out the practice of documenting things, let's - also put this here; although now that I think about it, I'm pretty - sure I've dumped this somewhere else before. - - The preferred method of loading large programs among "Common Lisp - enthusiasts" is [Quicklisp][quicklisp], which is a sort of apt-get - for CL, with centralized repositories and all that jazz. I've - never used it, incidentally; and it's not that I'm denying its - quickness or usefulness, but that process of automatically - fetching dependencies from some arbitrary site obscures my - understanding of the programs that I'm running and their real - mass. Instead, I prefer going through the laborious job of writing - down the entire dependency tree, then grabbing a copy of each - dependency from the author's site\*, putting them all in a - directory and defining the path to that in my CLtron - instance. Here's how this looks for Hunchentoot: - - ~~~~ {.commonlisp} - (defvar *ext-dep-base* "/home/spyked/lisp-stolen/") - (defvar *ext-deps* '("chunga/" "trivial-gray-streams/" "cl-base64/" - "cl-fad/" "bordeaux-threads/" "alexandria/" - "cl-ppcre/" "flexi-streams/" "md5/" "rfc2388/" - "trivial-backtrace/" "usocket/")) - ~~~~ - - then I'll define a variable holding the path to my - work-in-progress Hunchentoot code base: - - ~~~~ {.commonlisp} - (defvar *hunchentoot-path* "/home/spyked/tmsr/hunchentoot/b/hunchentoot/") - ~~~~ - - then I'm making sure I get rid of some useless dependencies, - e.g. SSL: - - ~~~~ {.commonlisp} - (pushnew :drakma-no-ssl *features*) - (pushnew :hunchentoot-no-ssl *features*) - ~~~~ - - and now I have to instruct [ASDF][asdf] to look for "systems", - i.e. Common Lisp programs, in each of the directories in the paths - above. Apparently we're not quite at the point where we can get - rid of this particular piece, so: - - ~~~~ {.commonlisp} - (loop for path in *ext-deps* do - (pushnew (concatenate 'string *ext-dep-base* path) - asdf:*central-registry* - :test #'string=)) - (pushnew *hunchentoot-path* asdf:*central-registry* :test #'string=) - ~~~~ - - Oh, and by the way: - - ~~~~ {.commonlisp} - > (length *ext-deps*) - 12 - ~~~~ - - which are *all* the dependencies needed to run Hunchentoot given a - Linux-and-SBCL installation. At this point we can tell ASDF to - load our Hunchentoot: - - ~~~~ {.commonlisp} - (asdf:load-system :hunchentoot) - ~~~~ - - And after a second or so, we should be all prepped and ready to - start our web server. - - \-\-\- - \*: Not that this makes much of a difference, mind you. By now I - already have most dependencies commonly found in CL programs on - the disk, so I'm e.g. using whatever version of [usocket][usocket] - that I got whenever I got it from wherever. So as per the end of - the first footnote: since I'm already using that shit although I - haven't actually read the code, why haven't I published it - already? The man [makes a good point][btcbase-1924190], I *am* - using it. So how do I address the gray area of "I've been using - this piece of code for a while because my program requires it, but - I don't trust it enough to sign it just yet"? - -[^4]: My CLOS-fu is somewhat lacking, but this "next most specific - method" refers in principle to the method implementation of what - other languages call "the direct superclass", i.e. in our case the - acceptor class. This means that if our call to "/yo" doesn't - match, the server will fall back to the default mechanism of - serving static files from the document root. - -[cl-www]: /posts/y05/090-tmsr-work-ii.html#selection-108.0-108.17 -[hunchentoot-i]: /posts/y05/093-hunchentoot-i.html -[hunchentoot-ii]: /posts/y05/096-hunchentoot-ii.html -[wot]: http://wot.deedbot.org/ -[io]: /posts/y04/069-on-intellectual-ownership.html -[usocket]: http://archive.is/3UKXf -[btcbase-1923536]: http://btcbase.org/log/2019-07-19#1923536 -[cuntoo]: http://btcbase.org/log-search?q=cuntoo -[btcbase-1918973]: http://btcbase.org/log/2019-06-20#1918973 -[hunchentootarch-dot]: /uploads/2019/07/hunchentootarch.dot -[btcbase-patches]: http://btcbase.org/patches -[svg-use]: http://archive.is/JfGyb -[btcbase-1922361]: http://btcbase.org/log/2019-07-12#1922361 -[quicklisp]: http://archive.is/Bk8Rm -[asdf]: http://archive.is/oPKRp -[btcbase-1924190]: http://btcbase.org/log/2019-07-22#1924190 -[dependencies]: /posts/y03/04e-the-myth-of-software-engineering-iii.html#selection-85.0-87.0 -[ht-acceptor-dispatch-request]: http://coad.thetarpit.org/hunchentoot/c-acceptor.lisp.html#L628 -[ht-handle-static-file]: http://coad.thetarpit.org/hunchentoot/c-misc.lisp.html#L151 -[ht-document-root]: http://coad.thetarpit.org/hunchentoot/c-acceptor.lisp.html#L169 -[ht-docs]: http://archive.is/MP2bT -[ht-easy-acceptor]: http://coad.thetarpit.org/hunchentoot/c-easy-handlers.lisp.html#L330 -[ht-dispatch-easy-handlers]: http://coad.thetarpit.org/hunchentoot/c-easy-handlers.lisp.html#L319 -[cl-who-ii-fn4]: /posts/y05/095-cl-who-ii.html#fn4 -[ht-define-easy-handler]: http://coad.thetarpit.org/hunchentoot/c-easy-handlers.lisp.html#L164 -[mp-wp]: http://btcbase.org/log-search?q=mp-wp diff --git a/posts/y06/097-hunchentoot-iii.markdown b/posts/y06/097-hunchentoot-iii.markdown new file mode 100644 index 0000000..efd1c98 --- /dev/null +++ b/posts/y06/097-hunchentoot-iii.markdown @@ -0,0 +1,382 @@ +--- +postid: 097 +title: Hunchentoot: further architectural notes; and usage examples +date: July 26, 2019 +author: Lucian Mogoșanu +tags: tech, tmsr +--- + +This post is part of a series on [Common Lisp WWWism][cl-www], more +specifically a continuation of [ongoing work][hunchentoot-i] to +[understand][hunchentoot-ii] the web server known as Hunchentoot and, +as a result, produce a signed genesis to be used by members of the +TMSR [WoT][wot]. + +In this episode we'll do another illustration of the Hunchentoot +architecture; and we'll have fun documenting a running web server +instance, thusly exploring for now a few of things that it can do[^1]. + +First, the architectural diagram, with bells, whistles and clickable +stuff: + + +
+ + + +
+ +No, really, I'm not kidding: if you have a browser that implements +SVG, clicking on the text should direct you to defgeneric pieces of +code[^2]. Anyway, the big squares are Hunchentoot components and, more +specifically, the name of their CL classes, while the small squares +found inside the big ones represent methods specializing on a given +class. The green boxes are user actionable or defineable methods, so +this is where you should start looking; while the arrows denote the "X +calls Y" relation, with the exception of the dashed green arrow, that +tells us that header-out is in fact a setf-able accessor used from +somewhere within the context of acceptor-dispatch-request (e.g. from a +request handler) to read and modify the header of a reply. + +Now from this airplane view, Hunchentoot's organization looks quite +digestible, which should give us a very good idea of how to start +using it. So let's take a look at that, shall we? + +Assuming we've loaded[^3] Hunchentoot into our CLtron of choice, we +can now create an acceptor instance and start it: + +~~~~ {.commonlisp} +> (defvar *myaccept* + (make-instance 'hunchentoot:acceptor :port 8052)) +> (hunchentoot:start *myaccept*) +~~~~ + +... and now what? Say, for now, that we want to serve a static site -- +I'm using The Tar Pit as my playground, but you can use whatever you +fancy. Looking at +[acceptor-dispatch-request][ht-acceptor-dispatch-request], we notice +that it calls [handle-static-file][ht-handle-static-file] with the +[document-root][ht-document-root] as an argument. So let's set that, +and additionally the error template directory, to our site: + +~~~~ {.commonlisp} +(setf (hunchentoot:acceptor-document-root *myaccept*) + "/home/spyked/thetarpit/site/" + (hunchentoot:acceptor-error-template-directory *myaccept*) + "/home/spyked/thetarpit/site/") +~~~~ + +and now `curl http://localhost:8052` should serve its contents. + +But let's say we want to go one step further and serve some content +(server-side) dynamically. The original Hunchentoot +[documentation][ht-docs] actually provides a neat minimal example, +which I'm going to steal, but not before explaining what we're going +to do here. + +Besides serving files off the disk, a web server can do other useful +stuff, such as, in Apache's case, sending the file to a preprocessing +engine (PHP or whatever), or as we're going to show, executing some +other predefined action that depends on the request parameters (URL, +cookies, HTTP method, variables and so on). For now, let's say that we +want our server to respond to the URL "/yo" (where "/" is the site +root) with the plain-text message "Hey!". Furthermore, let's say that +we want to optionally parameterize requests to this URL by the +variable "name", in which case the response will include the name: for +example, if we do a GET request to "/yo?name=spyked", we want the +server to respond with "Hey, spyked!". + +We have a few possible ways of doing this. We could for example edit +the current implementation of +[acceptor-dispatch-request][ht-acceptor-dispatch-request], which is +also the ugliest possible approach. On the other hand, Hunchentoot is +built using Common Lisp's Object System mechanism (CLOS), which allows +us to subclass the acceptor to a user-defined class and specialize the +method above for our class. Let's try this out: + +~~~~ {.commonlisp} +(defclass myacceptor (hunchentoot:acceptor) + ()) +(change-class *myaccept* 'myacceptor) +~~~~ + +The change-class thing isn't something that we'd normally do, but if +you've been following along, you'll notice that this didn't break our +code, because well, Common Lisp is cool. Now for the dispatcher +method: + +~~~~ {.commonlisp} +(defmethod hunchentoot:acceptor-dispatch-request + ((acceptor myacceptor) request) + (cond + ((string= (hunchentoot:script-name request) "/yo") + (let ((name (cdr (assoc "name" (hunchentoot:get-parameters request) + :test #'string=)))) + (setf (hunchentoot:content-type*) "text/plain") + (format nil "Hey~@[, ~A~]!" name))) + (t (call-next-method)))) +~~~~ + +In human words: this is an implementation of acceptor-dispatch-request +specialized on myacceptor, that, upon encountering the URL +(script-name) "/yo", takes the value of the GET parameter known as +"name" and returns a response string (possibly containing this "name") +as plain text. Otherwise it transfers control to the "next most +specific method"[^4], implicitly passing to it the existing arguments. + +We could stop here, but we won't, as there's a short discussion to be +had, mainly related to the extensibility of our approach, i.e. what +happens when we add other custom URLs to this recipe? The naive result +will look ugly and will be a pain to maintain and debug; while the +more elaborate approach, involving putting every "/yo" into its own +function, will initially fill our implementation with cond/case +conditions, eventually leading to a more civilized dispatch mechanism, +in the form of a lookup table from URLs to handler functions. + +Well, it so happens that Hunchentoot already has an implementation for +this type of thing, going under the name of +[easy-acceptor][ht-easy-acceptor]. easy-acceptor defines a dispatch +table whose only dispatcher is (initially) the +[dispatch-easy-handlers][ht-dispatch-easy-handlers] function, which +looks up handlers for URLs in a global handler list, +\*easy-handler-alist\*. As things usually go with these +[domain-specific languages][cl-who-ii-fn4], most of the handler +maintenance work is piled up in the +[define-easy-handler][ht-define-easy-handler] macro. + +So, in order to illustrate this easy-stuff, first let's undo some of +our previous work and redo the very basics: + +~~~~ {.commonlisp} +(hunchentoot:stop *myaccept*) +(setq *myaccept* (make-instance 'hunchentoot:easy-acceptor + :port 8052 + :document-root "/home/spyked/thetarpit/site/" + :error-template-directory + "/home/spyked/thetarpit/site/")) +(hunchentoot:start *myaccept*) +~~~~ + +Notice how now we're instancing easy-acceptor. Now we can define an +equivalent "easy handler" for our previous "/yo" work: + +~~~~ {.commonlisp} +(hunchentoot:define-easy-handler (say-yo :uri "/yo") (name) + (setf (hunchentoot:content-type*) "text/plain") + (format nil "Hey~@[, ~A~]!" name)) +~~~~ + +which about sums up our exercise. Initially I had wanted to show an +example doing some fancy prefix/"smart" URL lookup à la +[MP-WP][mp-wp], but by now this post is so large[^5] that it can't be +eaten in one sitting. Alas, I will have to leave all my fancy examples +for another episode. Thus, until next time... + +[^1]: Contrary to popular beliefs and expectations, the things that + some particular X can do that are known to (some particular) me + are not to be confused with the total set of things that said X + can possibly do, nor with the set of things that it *can't* + do. Take for example X = your average pointeristic slash + buffer-overflowistic C barfola: you can identify some particular + uses for it, sure, but meanwhile the average douchebag will + exercise code paths that may make it usable for things you've + never imagined, such as stealing your keys, wiping your disk and + murdering your dog... and many other things, short of making you + some french fries, which is something that e.g. a web server can't + do. + + In other words, nobody gives a fuck about popular beliefs and + expectations; and by the time I publish a [signed][io] genesis for + this Hunchentoot thing -- good, bad, with or without warts or + however we have it -- I will be entirely able to say what it does + and doesn't do, which is exactly what I'm working on here and now. + + And now to be an asshole and leave this otherwise properly rounded + footnote hanging: what about, say, [usocket][usocket]? and then + what about SBCL or some other working CLtron? and what about + [Linux][btcbase-1923536] and [its userland][cuntoo]? This + unfortunately is the curse of our postmodern times: our ability to + run computing machines rests, for the time being, upon the promise + of some [shitheads][btcbase-1918973]. + +[^2]: I don't write HTML and CSS for a living, so I might as well use + this footnote to document the pain required to generate this, for + later reference. + + Specifying the diagram in GraphViz is fairly straightforward: one + simply has to list the clusters, the nodes and the edges within + them in a text file -- see for example the final [.dot + file][hunchentootarch-dot] used for generating the illustration + above. Adding links and colours and all that is also easy, as + previously shown. The problem, however, with this GraphViz thing + is that graph generation involves an automated step, i.e. node + layout generation and edge routing, that can easily prove to be a + pain in the ass for the user: not only do I want this diagram + generated, but I also want it to be arranged like *so*, and not + like *that*, because I want the viewer to be able to look at the + components of the graph in some particular order. + + To add insult to injury, this automated step is almost entirely + opaque to the user: in order to have that square near that one, I + need to frantically shuffle nodes and edges about until I find the + magic ordering that generates something close to what I want -- + that is, the relationship between said ordering and the output is + purely coincidental, and I'm stuck guessing based on the vague + hints found in the spec. Anyway, this is the best diagram layout + we've got here at The Tar Pit, sorry... do make sure to write in + if I'm in the wrong. + + Now that I have a representation, I need to embed it in the blog + post. One would expect that's also straightforward, wouldn't he? + Well, no! You see, I got the idea that placing clickable links in + generated SVG files is cool, only this doesn't work in the + slightest when inserting the "<img>" tag, because completely + counter-intuitively for a SVG, the browser displays *an image*, + not a DOM sub-tree. So then I look at how Phf did it with his + [patch viewer][btcbase-patches], and it looks like he's inserting + a HTML image-map in the HTML document, which kinda beats the + purpose of having links in the SVG in the first place. I really, + *really* don't want to copy-paste the whole diagram into the post, + so what the fuck am I gonna do, use <object> tags?! + + So if by now you were curious enough to look at the page source, + you'll notice that what I did was to insert an inline <svg> + that then imports the content of my .svg file using the + [<use>][svg-use] tag, which works exactly the way I want + it. And no, you won't find this anywhere on Google either, because + Google [doesn't fucking work][btcbase-1922361]. + + To sum this up: IMHO the result looks pretty cool, with the + mention that I'm most likely going to write the SVG diagram "by + hand" next time I'm doing anything non-trivial. At least then no + magic tool will lie to me that it saves hours of my work, when it + instead adds to it. + +[^3]: Since I'm trying out the practice of documenting things, let's + also put this here; although now that I think about it, I'm pretty + sure I've dumped this somewhere else before. + + The preferred method of loading large programs among "Common Lisp + enthusiasts" is [Quicklisp][quicklisp], which is a sort of apt-get + for CL, with centralized repositories and all that jazz. I've + never used it, incidentally; and it's not that I'm denying its + quickness or usefulness, but that process of automatically + fetching dependencies from some arbitrary site obscures my + understanding of the programs that I'm running and their real + mass. Instead, I prefer going through the laborious job of writing + down the entire dependency tree, then grabbing a copy of each + dependency from the author's site\*, putting them all in a + directory and defining the path to that in my CLtron + instance. Here's how this looks for Hunchentoot: + + ~~~~ {.commonlisp} + (defvar *ext-dep-base* "/home/spyked/lisp-stolen/") + (defvar *ext-deps* '("chunga/" "trivial-gray-streams/" "cl-base64/" + "cl-fad/" "bordeaux-threads/" "alexandria/" + "cl-ppcre/" "flexi-streams/" "md5/" "rfc2388/" + "trivial-backtrace/" "usocket/")) + ~~~~ + + then I'll define a variable holding the path to my + work-in-progress Hunchentoot code base: + + ~~~~ {.commonlisp} + (defvar *hunchentoot-path* "/home/spyked/tmsr/hunchentoot/b/hunchentoot/") + ~~~~ + + then I'm making sure I get rid of some useless dependencies, + e.g. SSL: + + ~~~~ {.commonlisp} + (pushnew :drakma-no-ssl *features*) + (pushnew :hunchentoot-no-ssl *features*) + ~~~~ + + and now I have to instruct [ASDF][asdf] to look for "systems", + i.e. Common Lisp programs, in each of the directories in the paths + above. Apparently we're not quite at the point where we can get + rid of this particular piece, so: + + ~~~~ {.commonlisp} + (loop for path in *ext-deps* do + (pushnew (concatenate 'string *ext-dep-base* path) + asdf:*central-registry* + :test #'string=)) + (pushnew *hunchentoot-path* asdf:*central-registry* :test #'string=) + ~~~~ + + Oh, and by the way: + + ~~~~ {.commonlisp} + > (length *ext-deps*) + 12 + ~~~~ + + which are *all* the dependencies needed to run Hunchentoot given a + Linux-and-SBCL installation. At this point we can tell ASDF to + load our Hunchentoot: + + ~~~~ {.commonlisp} + (asdf:load-system :hunchentoot) + ~~~~ + + And after a second or so, we should be all prepped and ready to + start our web server. + + \-\-\- + \*: Not that this makes much of a difference, mind you. By now I + already have most dependencies commonly found in CL programs on + the disk, so I'm e.g. using whatever version of [usocket][usocket] + that I got whenever I got it from wherever. So as per the end of + the first footnote: since I'm already using that shit although I + haven't actually read the code, why haven't I published it + already? The man [makes a good point][btcbase-1924190], I *am* + using it. So how do I address the gray area of "I've been using + this piece of code for a while because my program requires it, but + I don't trust it enough to sign it just yet"? + +[^4]: My CLOS-fu is somewhat lacking, but this "next most specific + method" refers in principle to the method implementation of what + other languages call "the direct superclass", i.e. in our case the + acceptor class. This means that if our call to "/yo" doesn't + match, the server will fall back to the default mechanism of + serving static files from the document root. + +[^5]: Around 2300 words to be more precise, current footnote excluded; + of which the post body weighs a bit over one thousand, while the + footnotes contain a bit over one thousand and two hundred. And + look, footnotes 1 and 2, which grew organically out of elaboration + and documentation requirements, could have been posted on their + own, as a separate article each. + + On the other hand this is why I call them notes, so that I don't + spend more time moving stuff around than I do writing. The reader + will just have to live with my peculiar way of organizing + thoughts. + +[cl-www]: /posts/y05/090-tmsr-work-ii.html#selection-108.0-108.17 +[hunchentoot-i]: /posts/y05/093-hunchentoot-i.html +[hunchentoot-ii]: /posts/y05/096-hunchentoot-ii.html +[wot]: http://wot.deedbot.org/ +[io]: /posts/y04/069-on-intellectual-ownership.html +[usocket]: http://archive.is/3UKXf +[btcbase-1923536]: http://btcbase.org/log/2019-07-19#1923536 +[cuntoo]: http://btcbase.org/log-search?q=cuntoo +[btcbase-1918973]: http://btcbase.org/log/2019-06-20#1918973 +[hunchentootarch-dot]: /uploads/2019/07/hunchentootarch.dot +[btcbase-patches]: http://btcbase.org/patches +[svg-use]: http://archive.is/JfGyb +[btcbase-1922361]: http://btcbase.org/log/2019-07-12#1922361 +[quicklisp]: http://archive.is/Bk8Rm +[asdf]: http://archive.is/oPKRp +[btcbase-1924190]: http://btcbase.org/log/2019-07-22#1924190 +[dependencies]: /posts/y03/04e-the-myth-of-software-engineering-iii.html#selection-85.0-87.0 +[ht-acceptor-dispatch-request]: http://coad.thetarpit.org/hunchentoot/c-acceptor.lisp.html#L628 +[ht-handle-static-file]: http://coad.thetarpit.org/hunchentoot/c-misc.lisp.html#L151 +[ht-document-root]: http://coad.thetarpit.org/hunchentoot/c-acceptor.lisp.html#L169 +[ht-docs]: http://archive.is/MP2bT +[ht-easy-acceptor]: http://coad.thetarpit.org/hunchentoot/c-easy-handlers.lisp.html#L330 +[ht-dispatch-easy-handlers]: http://coad.thetarpit.org/hunchentoot/c-easy-handlers.lisp.html#L319 +[cl-who-ii-fn4]: /posts/y05/095-cl-who-ii.html#fn4 +[ht-define-easy-handler]: http://coad.thetarpit.org/hunchentoot/c-easy-handlers.lisp.html#L164 +[mp-wp]: http://btcbase.org/log-search?q=mp-wp