• Plain Text
  • 455 lines
  • 8640 views
  • 7 forks
  • Pasted by anonymous on August 13, 2013
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
Okay cool so, we need to standardize something of an API for route-handlers (the callbacks assigned to routes in Compojure).

At present it's a given that the Ring request map/object has to be propagated throughout the app (I've already written the frontend macro for this).

(defmacro render-template [filename context]
  `(render-template-fn ~filename (assoc ~context :request ~'request)))

This is the frontend macro and leans on the same /the request is always available by the name "request"/ contract.

The problem is, and I realize this might be a trivial complain, is that almost every handler is going to have to reach into (request :params).

At present my Compojure routes are destructuring and passing params *AND* request into each handler, which I'm not totally happy about.


(defroutes admin-routes
  (GET "/admin/edit/page/:id/" {params :params :as request} (superuser-only edit-flatpage request params)))

(defn edit-flatpage [request params]
  "Edit a flatpage"
  (let [id (:id params)
        flatpage (get-document-by-id "flatpages" id)]
    (make-flatpage request (merge {:edit true} flatpage))))

I'd really prefer to keep it single-arity with everything in the request map at the handler level unless something truly compelling comes up later. I would also prefer not to manually destructure params out at the handler level over and over.

I'd also like to keep route handlers a vanilla (defn ...). I don't mind macros, but I'd rather they not take over the whole handler body. That invites too many demons.

I know these are probably persnickety sounding prequisites but consider what Flask's API looks like with a much less powerful language:

@app.route("/")
def hello():
    return "Hello World!"

In python, def defines a plain ole' function.

Options being considered are:

A macro that does:

(param :id) => (:id (:params request))

I know it's undesirable to trick the user into thinking param is a map. One alteration that could help here is:

(get-param :id) ;; which makes it seem more like a function. (They're going to be importing the macro anyway)

Thoughts?
so in python how do you get params/req?

the example doesn't bother with them no?

No it doesn't, in Flask you'll do:

request.args.get("id") or request.form.get("id")

in Django it's:

request.POST.get("id") or request.GET.get("id")

--

so that's not really any better no?

You're right it's not, but I think we can do better than having the same destructuring patterns over and over in Clojure. Without necessarily engaging in undue evil. To illustrate my point:

(defn handler [request]
  ;; Note that Compojure params unifies form encoded, url encoded, GET args all in one, unlike Flask/Django.
  (let [params (request :params)
         id (:id params)])))

so why not just do that

you have

(defn handler [f]
  (fn [request]
    (let [params (:params request)]
      (f request params))))


(GET "/foo/" req (handler request my-handler))


I forgot to explain one part. I want a consistent arity/contract across all handler functions if possible. fn accepts one arg, "request". I want brevity in the route definitions too.

wouldn't it be better to accept request and params?

then you can use whichever one you care about or both, I mean 2 args is not crazy :)

It's not crazy, but it's redundant. The information is 100% in the request object and it's unnecessary. I have to firm up and decide on this API because I am writing that macro to clean up the messy Compojure routes. I'd like to make certain that decision is elegant.

 

but I mean that's the default behavior no?

(GET "/foo/" req my-fn)

my-fn takes the req, I'm not sure I follow the difference :)

That's not really default. Default is no args, (GET "/foo/" [] my-fn) -- and I need to be able to define preprocessor fns for handlers without repeating myself.
I'd like to be able to do arbitrary sub-groupings as well. the req -> request handoff isn't actually that well documented in Compojure's notes on destructuring.

Easy enough to see why I'd like this given: https://github.com/bitemyapp/neubite/blob/master/src/neubite/routes/admin.clj#L100-L111

The only useful information per line is (GET "route/" handler-fn) superuser-only, the destructuring pattern, and the manual request/params passing is all redundant.

and yes, I could clean up the destructuring pattern a little bit, but it's not good enough. Yes?

Remember we're competing with:
  @app.route('/login', methods=['POST', 'GET'])

 g hold up


it feels like you could clean that up by making superuser-only behave differently, more like

(defn handler [f]
  (fn [request]
    (let [params (:params request)]
      (f request params))))


No me gusta. Bad separation of concerns. superuser-only shouldn't care about destructuring things for the request.

I realize your suggestions would help, but they only get us half of the way there. I need to get from:

(POST "/admin/write/flatpage/" {params :params :as request} (superuser-only make-flatpage request params))

;; to

(POST "/admin/write/flatpage/" make-flatpage) ;; with a :preprocessors [superuser-only] at the top

Yes I looked at Compojure context.

also have you looked at liberator? :)

http://clojure-liberator.github.io/liberator/tutorial/post-et-al.html


(resource
        :allowed-methods [:post :get]
        :available-media-types ["text/html"]
        :handle-ok  ...)

if you want to get really restful that's one way to go

I've looked at Liberator. It has *some* promise but there are a few problems I have with it. For one thing, it reminds me WAY too much of Django Class-based views (used to be functions only). Django CBV has been a nightmare. For another, liberator only kinda sorta solves the problem I want to solve, and it brings in a bunch of baggage that I don't really care for.

I really just want to write a macro for cleaning up the routes. It really is scoped to just that and I don't see scope creep of the scale Liberator achieved happening. Not yet anyway. I could be convinced otherwise, but for now, no.

well I'm personally not a huge fan of the liberator approach, I tried it and I find it can get confusing fast
the decision graph is fiddly

As somebody who's had to work on the kind of problems that typically commend the use of graph libraries and databases, I no longer trust people that recommend using anything involving graphs. Unless it's version control. I don't like the idea of graph-spaghetti in my goddamn webapp. I've worked on HUGE applications and not needed that.

Basically, all I've really ever needed and has been a help 90% or more of the time has been:

Middleware (donesies. Thanks Ring)

Routing      (Compojure! regex is nice sometimes.)

Signals (Sync and Async. Don't really have a disciplined approach to this yet in Clojure)

Async Tasks ^^ (feeds off Signals work. separate worker server cluster feeding off same persistence layer, with a different app init is ideal)

Persistence abstraction (we can do better here than Korma.)

Clean accessing of inputs (params)

Sensible templating with proper inheritance (Selmer! We're good here :))))))))

Pre/post handlers. This'll come with the routing macro cleanup.

That's...about it. Everything else is made pretty easy by having the above. For example, superuser-only is fundamentally ACL. If using pre-handlers is lightweight and easy, then so is your ACL layer. If the "request" object is the universal way to get request processing information, then your ACL layer can just standardize on that and you can go home happy.

Make sense? Also I'm considering adding an "auto destructure" parameter to the macro for auto-injecting data from the request into the template in addition to my little user hack.

it does, so the main thing to identify atm is what exactly you want to be able to write in your route def
I thought I did above. Let me make a complete example.

(defroutes admin-routes
  (bitemyapp-magic "/admin/" :pre [superuser-only] :post [admin-renderer]
    (GET/POST "edit/page/:id/" edit-flatpage)))

(defn edit-flatpage [request]
  (let [id (get-param :id)]
    ;; do stuff
    ;; return success/fail, now the admin-renderer handles redirect/template invocation. This is to prevent duplication referencing of the template path.))


Boom. that. And the more routes/pre/post involved, the more efficient and easy to read this becomes compared to the current default.

Notice I reused the Compojure context thing for "/admin/" prefixing. I'm considering a postfixer as well but leaving it alone for now.

ok so now that we know what we want
I definitely like the :pre and :post idea

right

I like working like this. It's not perfect by any means, but it's nice.

Another design parameter is that people should be able to work directly with any of the underlying sub-structure without anything breaking. That means Compojure, Ring middleware, whatever. So if they don't use the bitemyapp-magic macro, it works like a vanilla defroutes.

right and it has to play nice with other macros

And it will. I have a provisional macro for this that already plays nice with defroutes. It generates the vanilla format and you can macroexpand it to check output.

Also I shared this link with Raynes, hoping for some thoughts from him since you've worked with him on lib-noir.

Another note on liberator, their use of (dosync ...) fucking terrifies me. I just went to the trouble of cleaning up thread-local state in neubite, I'm not about to invite the vampire back in.

I'm trying to follow what you want to accomplish here and having a tiny bit of trouble. Are you trying to access 'params' in a handler without taking it in as an argument? Like, would this 'params' macro be anaphoric or something?

Well maybe. There's a greater goal than that. The point is to start DRY'ing up Ring/Compojure code with a basic contract. That contract being, "handlers always have the request available and accept that one argument". The only reason params came up is because a lot of handlers are going to need it and I don't want repetitious destructuring. The aforementioned contract actually makes this something of a non-problem.

This is a design discussion, nobody present has any trouble implementing this.


Right, your macro works only if the request argument is always named the same thing.

Yep. I'm okay with that. There's no reason to name it anything else. The standardization is more useful than the freedom implied by having request vs. req.

Just an FYI if you missed it - Pedestal relies on the exact same contract.


Pedastal is the devil.

Symbol capture is pretty scary. Everyone is going to hate you for it. If you're okay with that, then go ahead. :p

They already hate me anyway. I know what I'm doing is distasteful to some, but I don't care.


I'm personally not a huge fan of capturing var names either

I'm having this discussion in the hopes of being presented with a viable alternative. It has to have the same brevity, clarity, and regularity of interface tho.


IW tehinl, e only real alternative I can think of is to simply have these sorts of things be functions and pass request into them -- obviously this is the opposite of what you want, though. I have no viable alternatives.

I'm not really too bothered by

(GET "/foo" req ...)

Did you see my earlier examples? it gets hairier than that quickly.


but the handler wrapper could deal with that no?

(GET "/foo" req (handler ...))
I
 assume you don't want to write out handler every time?

Correct. We covered :pre and :post in the defroutes inner-macro. Duplication isn't really acceptable. Passing the request object to an fn doesn't solve my repetitious destructuring problem. I'm already doing the symbol capture thing in my view-handling macro.

I don't even like the manual "req" in the middle of the route. It's a given that a request would be passed. I shouldn't need to specify unless I'm changing the behavior.





sooo

I would vote for a macro, I know you said you want to keep defn for handler

but I think might actually be nicer to make a defhandler

and then it can do :pre, :post business and what not

but yeah why not a defhandler?

so you'd have


(defhandler my-handler [])

(GET "/foo" req my-handler)

the defhandler could then expand the req, specify pre post, etc
have its own protocol


So, you're going to have to specify that a little more in terms of what you think that would look like, but so far that violates a few design parameters I laid out above. I *really* don't want to slam new people with a def handler macro. Really really really. Really. I'd rather just mandate the (defn hndlr [request] ) contract.

Further, the point of the route definitions macro is to eliminate repetition. If I am manually specifying :pre and :post for a group of routes that all need the same :pre and :post I am going to be mad. :P

The defhandler would solve only the params break out problem and in a way that is less nice IMHO

I'm trying to hold onto handler macros as a last resort for when some *REAL* magic is needed. This doesn't justify it yet.

I'm not trying to hulk-smash your suggestions, I just haven't seen anything that satisfies the parameters yet. Is symbol capture really going to get peoples' hackles up if it's part of a contract?

But I'm not sure why a macro is any worse than any other contract
as long as it's simple and behaves intuitively there's not much difference
and you don't have to use it :)
until you get fed up with writing stuff out you can always write things the long way

I think the handler macro could deal with pre/post conditions nicely and break out the params, and possible do other stuff like allow optional restirciton params by request type, eg:

(defhandler handler [req :methods [:GET :POST] :pre whatever :post whatever])

(ANY "/foo" req handler)

That's partly why I like this design and why I want to keep the macros constrained and focused on one problem at a time. Like destructuring params. Making them all part of a unified "there must be a request symbol defined and pointing at a map" contract.

I'd probably end up doing the above more like:

(bitemyapp-magic "prefix/" :pre [whatever] :post [post-whatever]
  (ANY "/foo" handler))

(defn handler [request]
  ...)

have a bunch of keyword params for all kinds of stuff
I think it would be much more readable


eh I used to do that with restricted
it runs into problems playing with other middleware and macros

you pretty much want to leave (GET "/" ...) alone
weavejester feels pretty strongly about it :)

he convinced me of his wisdom

and I think pre/post would be fns no?

pre/post are fns, yes. I don't see what the problem is wrapping a universal context macro around the routes for duplicate information.

Because there might be more than one dude. :P

I'm serious, I have multiple (almost stacks of them) decorators for handlers in my more serious Python code all the time. Serving pre and post needs.

stuff like:

@json_api
@requires_superuser
@requires_group("SEXY")
@uses_template_lib
def handler(request):
return {"LEL": "LEL"}


so why put them in []?

well context macro breaks for one

I honestly think that defhandler macro addreses a lot of the problems

a) you don't need to write GET, POST, etc separately
b) you and add pre/post handling
c) extract params

It doesn't because it doesn't help me eliminate the grouping duplication. And I don't think it's information that should necessarily live with the defn.
I do *NOT* want to be defining superuser-only across 10 defhandlers. *do not want*

It mean this would be very similar no?

you do your defhandler

and then
(ANY "/" req handler)

lol so defdefhandler? :P

that would setup common shit and then make handlers?

You'd probably have more luck explaining to me what's wrong with Compojure's context macro. Because I'm following that for a guideline. You're going to have a really hard time convincing me on defhandler when it doesn't solve half the problems I want and introduces one I wanted to avoid.

well being a macro it can solve any problem :P
aside from that of being a macro :P

but the more I think about it the more it sounds like you want a handler templating mechanism

you want to be able to define a template like this page is restricted, it does this pre and post shit, and etc

then use that to make these kinds of pages no?

Close enough. A reminder of what my routes in Neubite look like right now: https://github.com/bitemyapp/neubite/blob/master/src/neubite/routes/admin.clj#L100-L111

That's fuckin' ick.

Just as a reminder guys, the full Flask example application, in a language with NO macros is:

from flask import Flask
app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello World!"

if __name__ == "__main__":
    app.run()

That covers things like gathering the routes and running the app and everything. Automatic route-gathering is probably something I should write too.

yeah so templating thing wouldn't meed macros, you just have a closure, initialize it with whatever and then use its output to create your handlers

Not really. I need to clean up the route definitions too. I feel like I keep switching between design parameters A, B, C, D, and E. I explain one or two and why something doesn't work for the other three, then we trade which ones we're ignoring.

I provided an example of what I wanted above. And an example of problematic code as it exists now.

here's the thing we really need to get some examples going
of shit the way it's done now

then the way we'd like it

yes but that one isn't really so bad that's the thing

the only thing right now that I think is a problem is that you have to be repetitive
like you said with the admin access being specified everywhere
or pre post conditions

that I grok :)

that's bad


but what's the other big issues really?

and gotta run here in a min

Hum. I listed them out, but I'll let ya go for now. I think it's best I just implement all of it and show you the final result so you can appreciate what I'm going for. I'll need to ask one or both of you about how Noir registered routes. I'd like to do so once at app init, do it cleanly'ish and call it a day.


I think so :)

do not do that!
what noir did was horrible!

Well tell me how ibdknox did it and I can think of a better way. Manually require'ing and composing routes is annoying.

magic global vars
and on top of that you had to make sure that all the namespaces were referenced in the handler for it to work or it would fail horribly and in mysterious ways when getting packaged


Welp. lol. Maybe I can sugar/clean up the right way of doing it then so that the "hello-world" app is cleaner.


I actually like all routes being in one place

when I look at the app I can go and see where all the entry points are

when I did it the noir way it was actually more difficult to tell

I grok ya. I agree. I'm probably going to be making some more experiments inside of Neubite and eventually take the patterns and macros and turn them into a somewhat cleaner hello-world app for educational purposes. I'm actually still poking at something inside http-kit that I'm concerned about anyway.



yeah that's a good plan, that's what I usually do as well
you have to play around with it till you figure out what actually works well
I like to throw shit at the wall till something sticks :P

I'm the same way.

and I'll leave you with that mental imagery :P
ping me later, I'll probably play around with this now too at some point

Did you know? CLOSE

  • There are keyboard shortcuts!
    • When Creating A Paste
      • ALT+P Toggle Private
      • CTRL+Enter Create Paste
      • ALT+W Toggle word wrap
    • When Viewing A Paste
      • ALT+G Go to a line
      • ALT+CTRL+E Edit the paste
      • ALT+R Show the raw code
  • There are URL options!
    • When Creating A Paste
      • ?lang=Javascript to default to javascript
    • When Viewing A Paste
      • #L-N Jump to line number N
?