00:53:32
| <rowbit> | /!\ ATTENTION: (default-local) [email protected] successfully signed up for developer browserling plan ($20). Cash money! /!\ |
00:53:32
| <rowbit> | /!\ ATTENTION: (default-local) paid account successfully upgraded /!\ |
00:54:06
| <substack> | yay |
00:54:10
| <substack> | cash dolores |
00:54:33
| * _sorensen | quit (Quit: _sorensen) |
02:10:14
| * tphummel | quit (Quit: tphummel) |
02:32:33
| * wiwillia | joined |
03:06:02
| * wiwillia | quit (Ping timeout: 244 seconds) |
03:17:21
| <substack> | http://substack.net/images/keeping_it_surrealtime.svg |
03:37:49
| <niftylettuce> | Please upboat on HN "Show HN: Chrome Extension to automatically block the annoying Quora login prompt" |
03:38:09
| <niftylettuce> | substack chapel pkrumins paul_irish dominictarr dools AvianFlu rook2pawn tanepiper ryanseddon |
03:38:13
| <niftylettuce> | :D QUORA POOOOP |
03:38:13
| <LOUDBOT> | FOX NEWS: OBAMA LEGISLATES MANDATORY ABORTION, HOMOSEXUALITY; DECLARES BURNING BIBLES AS A CORE COMPONENT OF NEW ENERGY PLATFORM |
03:38:55
| <AvianFlu> | LOUDBOT: twitlast |
03:38:56
| <LOUDBOT> | AvianFlu: http://twitter.com/loudbot/status/245365534487629824 (adipocere/#mefi) |
03:41:05
| * dools | likes his lettuce nifty |
03:41:25
| <niftylettuce> | GIGGITy |
03:41:26
| <LOUDBOT> | LET'S IMAGINE THIS: YOU HAVE TWO PEOPLE THAT ARE BOTH EATING A HOT DOG IN NY |
03:42:21
| <substack> | the niftiest lettuce |
03:42:37
| <substack> | I'm going to keep hurling art at people until everything looks awesome |
03:47:56
| <niftylettuce> | :D :D |
03:48:01
| <niftylettuce> | substack: :) <3 |
03:54:41
| <niftylettuce> | thx all |
03:55:18
| <chapel> | http://skeu.it/post/25597190946/via-alf-where-it-all-began-via-perezd-its |
04:10:07
| * st_luke_ | joined |
04:16:28
| * st_luke | quit (Disconnected by services) |
04:16:30
| * st_luke_ | changed nick to st_luke |
04:16:58
| * st_luke_ | joined |
04:28:49
| <substack> | getting some hella chinese leveling up |
04:29:03
| <substack> | before heading out |
04:31:08
| <AvianFlu> | substack: ARE YOU ALREADY IN THE BIG DATA CHINA MARKET!? |
04:31:19
| <AvianFlu> | damn, I spoilt my own loudbot fun |
04:31:21
| <substack> | YES INDEED |
04:31:22
| <LOUDBOT> | GET THE FUCK OUT OF MY WAY BEFORE I BEAT YOU SENSELESS WITH MY FUCKING NUTS |
04:31:24
| <st_luke> | CHINAMARKET |
04:31:25
| <LOUDBOT> | WHICH IS A KNOWN ISSUE WITH CRAZY PEOPLE |
04:31:31
| <substack> | 是的,我 |
04:31:41
| <AvianFlu> | WTF LOUDBOT THOSE WERE TOTALLY CAPS |
04:31:42
| <LOUDBOT> | LOL THAT'S A TERRIBLE IDEA RETARD |
04:31:52
| <st_luke> | LMBO |
04:32:07
| <substack> | 我准备到大的数据对整个中国市场 |
04:32:23
| <dools> | CHINESE CHARACTERS MAKE MY TERMINAL GO FUNNY. IT'S A CONSPIRACY |
04:32:24
| <LOUDBOT> | ACHIEVEMENT UNLOCKED: ACHIEVEMENT CERTIFICATE |
04:33:37
| <substack> | AND AWAY I GO BITCHES |
04:33:38
| <LOUDBOT> | I WAS TYPING STARTX WHEN CLINTON WAS PRESIDENT, BITCH. |
04:34:12
| <AvianFlu> | LOUDBOT: twitlast |
04:34:12
| <LOUDBOT> | AvianFlu: http://twitter.com/loudbot/status/245379444481601536 (Dav3xor/#dongtown) |
04:43:18
| * st_luke_ | quit (Disconnected by services) |
04:43:47
| * st_luke_ | joined |
04:52:57
| <st_luke> | got really excited about seeing a libdb driver in npm |
04:53:04
| <st_luke> | but it was just for libdbus |
05:00:49
| <st_luke> | hooray https://github.com/mcavage/node-bdb |
05:01:46
| <guybrush> | seems like not maintained and no gyp |
05:02:47
| <st_luke> | yep |
05:03:02
| <st_luke> | barely any js in it anyway |
05:11:55
| * mikeal | quit (Quit: Leaving.) |
05:12:45
| * mikeal | joined |
05:45:32
| * st_luke | quit (Remote host closed the connection) |
05:46:36
| * dominictarr | quit (Ping timeout: 240 seconds) |
06:25:38
| <guybrush> | sweeeeeeeeeet https://github.com/mranney/node_redis/blob/master/package.json#L2 |
06:25:48
| <guybrush> | with superfast js-parser |
06:39:03
| * st_luke | joined |
06:54:13
| * idletom | quit (Ping timeout: 268 seconds) |
07:01:28
| * substack | at sfo |
07:06:04
| * st_luke | quit (Remote host closed the connection) |
07:14:02
| * tilgovi | quit (Remote host closed the connection) |
07:51:19
| <rowbit> | /!\ ATTENTION: (default-local) paid account successfully upgraded /!\ |
07:51:20
| <rowbit> | /!\ ATTENTION: (default-local) [email protected] successfully signed up for developer browserling plan ($20). Cash money! /!\ |
07:53:05
| <guybrush> | substack: what do you think about making browserify().require('child_process').spawn() creating a webworker? |
07:54:17
| <guybrush> | oh maybe this is wrong when i think more about it |
07:56:18
| <guybrush> | send and .on('message') could be mapped to postMessage and worker.on('message' |
07:56:59
| <guybrush> | but everything else would behave very different :p |
07:57:29
| <substack> | yay |
07:57:58
| <guybrush> | btw webkitPostMessage is so fucking sexy |
07:58:37
| <substack> | sweets |
07:58:45
| <substack> | guybrush: make might sense to add that to child_process |
07:59:01
| <substack> | although it reads from files |
07:59:04
| <substack> | so that's a bit tricky |
07:59:15
| <substack> | can always do a separate module for it |
07:59:23
| <guybrush> | yes |
07:59:27
| * substack | boarding very soon |
07:59:37
| <guybrush> | oh have fun! |
07:59:58
| <guybrush> | if there are some slides, i cant wait to see them :p |
08:30:58
| <AvianFlu> | LOFTY PRIORITIES INDEED |
08:30:59
| <LOUDBOT> | I'M FLYING TO BOSTON TODAY MAYBE I WON'T DIE EVEN |
08:47:22
| * ITpro | quit (Read error: Connection reset by peer) |
08:48:27
| * ITpro | joined |
09:15:16
| * ITpro | quit (Ping timeout: 260 seconds) |
09:20:58
| * AvianFlu | quit (Quit: AvianFlu) |
12:31:33
| * timoxley | joined |
13:04:55
| * st_luke | joined |
13:15:08
| * idletom | joined |
13:16:21
| * dominictarr | joined |
13:32:32
| * idletom | quit (Ping timeout: 248 seconds) |
13:34:42
| * st_luke | quit (Remote host closed the connection) |
14:19:24
| * st_luke_ | changed nick to st_luke |
14:38:03
| * wiwillia | joined |
14:43:08
| * dominictarr | quit (Quit: Leaving) |
14:43:24
| * dominictarr | joined |
14:46:06
| * timoxley | quit (Quit: Computer has gone to sleep.) |
15:03:04
| * AvianFlu | joined |
15:08:39
| * _sorensen | joined |
15:13:06
| * timoxley | joined |
15:20:45
| * juliangruber | joined |
15:31:33
| * _sorensen | quit (Remote host closed the connection) |
15:41:49
| * juliangruber | quit (Ping timeout: 245 seconds) |
15:46:24
| <dominictarr> | Raynos, new flights booked! coming to london! |
15:47:46
| <Raynos> | sweet |
15:48:09
| <Raynos> | dominictarr: If you land during the day I may come meet you in london |
15:49:08
| <dominictarr> | Raynos, I land 18:30 LHR |
15:50:48
| <Raynos> | cant make that |
16:18:20
| * wiwillia | quit |
16:45:44
| * dominictarr | quit (Ping timeout: 276 seconds) |
17:02:15
| * mikeal | quit (Quit: Leaving.) |
17:02:46
| * mikeal | joined |
17:36:11
| * idletom | joined |
18:17:38
| * _sorensen | joined |
18:18:32
| * mikeal | quit (Quit: Leaving.) |
18:39:52
| * dominictarr | joined |
18:58:33
| * py1hon | quit (*.net *.split) |
18:59:43
| * py1hon | joined |
19:07:34
| * mikeal | joined |
19:14:23
| * tilgovi | joined |
19:36:02
| * idletom | quit (Ping timeout: 246 seconds) |
19:47:39
| * simcop2387 | quit (Excess Flood) |
19:48:39
| * simcop2387 | joined |
19:48:54
| * mikeal | quit (Quit: Leaving.) |
19:58:12
| * dominictarr | quit (Remote host closed the connection) |
20:22:01
| * mikeal | joined |
21:09:23
| * ryan_stevens | joined |
21:09:38
| * tilgovi | quit (Ping timeout: 276 seconds) |
21:21:05
| <mbalho> | if anyone wants a streaming challenge fork this and make it purely stream https://github.com/maxogden/stenographer |
21:21:49
| <mbalho> | right now it buffers the entire input audio stream https://github.com/maxogden/stenographer/blob/master/index.js#L30 |
21:40:21
| * tilgovi | joined |
21:51:07
| * substack | in taipei |
21:51:30
| <dools> | you've got a real taipei personality |
21:52:46
| <substack> | seems pretty japanesque |
21:52:52
| * dominictarr | joined |
21:52:58
| <substack> | dominictarr: \o |
21:53:03
| * substack | in taipei |
21:54:03
| <dominictarr> | wow, sweet. |
21:54:22
| <dominictarr> | substack, how is the street-food? |
21:54:49
| <substack> | just at the airport |
21:54:58
| <substack> | and everything is closed since it's early |
21:57:30
| <substack> | and I don't have any NT$ |
21:58:32
| <substack> | 29.47NT$ = 1 USD |
21:58:41
| <substack> | so stuff is pretty cheap |
22:00:03
| <Raynos> | substack: Does marak still liev with you? |
22:00:20
| <substack> | yes |
22:00:58
| <dominictarr> | hes going to india soon. needs a holiday. |
22:01:33
| <dominictarr> | substack, I'd bet they'd take USD, near the airport. |
22:02:04
| <substack> | if they have credit card readers those do the conversion automatically |
22:02:33
| <dominictarr> | in Cambodia, there would be streetfood pretty much any time at all. |
22:02:54
| <dominictarr> | maybe there would be 4 hours when there is no street food. |
22:03:32
| <dominictarr> | but in the middle of the night, there would be kids walking around banging on this wood block thing. |
22:04:01
| <dominictarr> | first I thought they where just into music -- some of them where pretty funky -- but turned out they where selling noodles. |
22:06:11
| <substack> | some shops look open now |
22:06:24
| <substack> | getting some espresso I think |
22:06:41
| <substack> | actually I'll wait until my laptop is fully charged, just 30 more minutes |
22:07:05
| <dominictarr> | substack, you are waiting for a connecting flight? |
22:07:05
| <substack> | I intend to get lost in shanghai later today |
22:07:07
| <substack> | yep |
22:07:25
| <dominictarr> | substack, get lost ++ |
22:07:45
| <dominictarr> | maybe rent a bicycle |
22:07:53
| <substack> | good plan |
22:08:01
| <dominictarr> | how long are you in shanghai for? |
22:08:53
| <substack> | I leave on the 17th |
22:09:00
| <substack> | 5 days |
22:09:05
| <st_luke> | awesome |
22:09:36
| <st_luke> | substack: what airline did you fly out on? |
22:09:42
| <substack> | china airlines |
22:10:16
| <st_luke> | foreign airlines are fu |
22:10:17
| <dominictarr> | find a friendly looking tuktuk driver, and take his number, and make him your tuktuk driver for your stay. |
22:10:18
| <st_luke> | fun |
22:10:39
| <dominictarr> | thats what I will do next time I'm in Asia. |
22:10:54
| <st_luke> | dominictarr: good idea |
22:11:17
| <st_luke> | when I was in thailand I used the same motorcycle taxi driver a bunch of times and it made it way more fun |
22:11:24
| <dominictarr> | he'll take you to eat at all his relatives and friends resturants, etc. |
22:11:38
| <isaacs> | dominictarr, Raynos: So, i'm bumping into a problem with some of the things we're talking about with this readable stream api |
22:11:45
| <isaacs> | the problem is that there's too many things now |
22:11:51
| <Raynos> | isaacs: :D |
22:11:54
| <dominictarr> | haha |
22:12:07
| <isaacs> | i'm considering scrapping the low/hight water marks |
22:12:14
| <Raynos> | I don't use them myself |
22:12:22
| <isaacs> | Raynos: they're important for real life things |
22:12:29
| <dominictarr> | isaacs, that stuff should be in userland. |
22:12:30
| <isaacs> | you don't use them because yoer' dealing with synthetic streams almost exclusively |
22:12:36
| <isaacs> | dominictarr: well, tcp and fs is where you need it most. |
22:12:39
| <isaacs> | (ie, at all)( |
22:12:46
| <dominictarr> | you just want a really really simple base class. |
22:13:08
| <isaacs> | i want a class that will work for all the things in node, also |
22:13:09
| <Raynos> | isaacs: https://gist.github.com/c9d21023a2e62ebdaf54 |
22:13:23
| <dominictarr> | and then let subclasses inherit from it, or use intermediates that make sense for their particular cas. |
22:13:24
| <dominictarr> | e |
22:13:26
| <Raynos> | thats my implementation of multi party pipe if it helps |
22:13:53
| <dominictarr> | there are a bunch of types of streams, and each has slightly different reqs. |
22:14:04
| <isaacs> | Raynos: y reemit(stream, target, ["end"]) |
22:14:08
| <isaacs> | Raynos: what is that? ^ |
22:14:18
| <isaacs> | dominictarr: tcp and fs are all i really care about. |
22:14:33
| <dominictarr> | isaacs, those are quite different types. |
22:14:40
| <isaacs> | dominictarr: yes |
22:14:45
| <dominictarr> | fs is really simple. just read and write. |
22:14:46
| <isaacs> | dominictarr: but that covers basically all the possible use cases. |
22:14:53
| <isaacs> | dominictarr: well, fs is pretty complicated actually |
22:15:05
| <dominictarr> | no no no, duplex is the complexist |
22:15:15
| <isaacs> | because reads are necessarily async, and the output can never actually be flushed, but you ought to pretend it is flushed if ther's only one buffered. |
22:15:24
| <dominictarr> | then filter streams are different |
22:15:26
| <isaacs> | but yes, duplex is the complexist, for sure |
22:15:33
| <dominictarr> | zlib != duplex. |
22:15:35
| <isaacs> | filters are just a special case of duplex |
22:16:00
| <isaacs> | proof: net.createServer(function (sock) { sock.pipe(new Gzip()).pipe(sock) }) |
22:16:04
| <Raynos> | isaacs: thats a shitty implementation of `source.on("end", function () { target.end() })` |
22:16:11
| <dominictarr> | isaacs, that is what I thought at first. but duplex is like a telephone, and filter is like a meat grinder. |
22:16:27
| <isaacs> | dominictarr: in this case, then, a telephone can grind meat, also |
22:16:37
| <isaacs> | whatever you need to do for duplex, it turns out you need to do that for filters as well. |
22:16:50
| <isaacs> | a filter is: "Based on xyz input, produce abc output" |
22:17:00
| <isaacs> | a duplex is: "you get output and you put in input" |
22:17:17
| <isaacs> | the only diff is the causal connection, but a duplex stream might have that causal connection as well. |
22:17:22
| <dominictarr> | that is the idea of a meat-grinder. you are transforming the input. |
22:17:26
| <isaacs> | right |
22:17:31
| <Raynos> | isaacs: filter is far easier then duplex |
22:17:36
| <dominictarr> | no, duplex is communication with a black box. |
22:17:38
| <isaacs> | but you could be transforming 10lb of meat into 50lb of motor oil |
22:17:44
| <dominictarr> | you can send data to me... |
22:17:51
| <Raynos> | with duplex you need to handlers, one to handle input and do something with it. And one to watch on some other thing that generates output and then generate output |
22:17:55
| <isaacs> | gzipped output is not anything at all "like" its source |
22:17:58
| <dominictarr> | but I am crazy. I could say ANYTHING in response. |
22:18:05
| <Raynos> | with filter you have one handler, one that watches input and does something sync / async and then generates output |
22:18:07
| <dominictarr> | or nothing. |
22:18:19
| <isaacs> | again: filter is a special case of duplex. |
22:18:24
| <isaacs> | i'm not saying filter = duplex |
22:18:31
| <isaacs> | filter : duplex :: square : rectangle |
22:18:33
| <Raynos> | The specific thing about duplex is that it's basically two streams, a readable & writable joined together. Whether a filter is a single readable / writable stream |
22:18:48
| <dominictarr> | Raynos, well put. |
22:18:56
| <dominictarr> | it's like two independent streams. |
22:19:02
| <isaacs> | look, for any filter, you can have a duplex stream that is *exactly the same* as that filter stream |
22:19:10
| <Raynos> | isaacs: but that's overly complex |
22:19:18
| <isaacs> | is it? |
22:19:26
| <Raynos> | any RW stream can always be a duplex stream of a R stream and a W stream |
22:19:33
| <Raynos> | but a single RW stream is less complex then two streams |
22:19:45
| <isaacs> | it turns out that whatever you need to do for duplex streams, filter streams are a subset of that, and it's not much less of a subset, really |
22:19:56
| <dominictarr> | isaacs, the significant difference is that the pause behaviour is passed through from one side to the other in a filter. |
22:19:56
| <isaacs> | the *only* thing you don't have to handle with filters is output coming before you write() anythign |
22:20:11
| <Raynos> | the other thing is |
22:20:20
| <isaacs> | dominictarr: not necessarily immediately. |
22:20:42
| <dominictarr> | isaacs, no. not necessarily. but in most cases. |
22:20:50
| <isaacs> | show me a difference other than "The output is causally connected (somehow, unspecifically, asynchronously) to the input" |
22:21:00
| <isaacs> | "not necessarily but in most case" = "not a constraint" |
22:21:01
| <dominictarr> | you could have a fliter stream that just changes buffers, for example. |
22:21:22
| <Raynos> | isaacs: Does a filter stream imply filter1.pipe(filter2).pipe(filter1) is an infinite loop? |
22:21:26
| <isaacs> | dominictarr: you could also have a filter stream that outputs either 2 or 20 depending on whether date.now() % 2 = 0, every third write. |
22:21:36
| <isaacs> | Raynos: not necessarily |
22:21:42
| <isaacs> | but in most cases :) |
22:21:58
| <isaacs> | a tee is another kind of potential filter. |
22:22:06
| * idletom | joined |
22:22:14
| <isaacs> | it has output, and input, and the output is causally connected (somehow) to the input (in that it's identical) |
22:22:21
| <Raynos> | isaacs: a duplex stream does not suffer from duplex1.pipe(duplex2).pipe(duplex1) being an infinite loop |
22:22:25
| <isaacs> | or, forget about tee, consider cat |
22:22:33
| <isaacs> | Raynos: it might, though |
22:22:41
| <Raynos> | only if they are echo streams |
22:22:43
| <isaacs> | Raynos: depending on what the duplex is. |
22:22:51
| <isaacs> | Raynos: orif they're just a conversation. |
22:22:58
| <isaacs> | Raynos.pipe(Isaacs).pipe(Raynos) |
22:22:58
| <Raynos> | and an echo stream is a stupid duplex stream that behaves like a through stream |
22:23:05
| <isaacs> | that conversation could go on forever without ever repeating itself. |
22:23:13
| <dominictarr> | isaacs, there is definately a fuzzy line between duplex and filter. but I have found it simpler to build one base class for through streams, |
22:23:13
| <isaacs> | but you and i are not mere filters!! |
22:23:17
| <dominictarr> | and another for duplex. |
22:23:35
| <isaacs> | dominictarr: right, but what i'm saying is, from the pov of the interface, supporting one means you have to support the other. |
22:23:38
| <Raynos> | isaacs: I agree with dominic that base classes for through, write, read & duplex reduce complexity |
22:24:05
| <isaacs> | you cannot safely assume that you can just slap a filter(chunk) function that returns the transformed value (or even emits a transformed value later) and call it a day |
22:24:10
| <isaacs> | zlib is the canonical filter stream. |
22:24:17
| <isaacs> | and that approach is insufficient for zlib |
22:24:32
| <dominictarr> | isaacs, see http://npm.im/through |
22:24:38
| <isaacs> | another paradigm filter stream is crypto |
22:24:41
| <Raynos> | isaacs: do you want me to implement zlib as a through stream ? |
22:24:43
| <dominictarr> | that is my baseclass for filter streams. |
22:24:53
| <Raynos> | a filter stream does not have to be a synchronous filter |
22:25:26
| <isaacs> | dominictarr: so, the issue with zlib is that the output and input don't even have a 1:1 correllation |
22:25:33
| <dominictarr> | isaacs, it would be great if crypto was a stream. |
22:25:40
| <isaacs> | when compressing, you may write 10 times, and then get 1 chunk emitted |
22:25:48
| <dominictarr> | isaacs, whats the problem with that? |
22:26:13
| <isaacs> | dominictarr: in the other direction, you write one chunk, and the next write returns false, as it emits chunk after chunk of results until the buffer drains |
22:26:26
| <isaacs> | so, in practice: |
22:26:33
| <isaacs> | 1. the write() end needs to respect the false. |
22:26:47
| <isaacs> | 2. the read() end needs to be prepared to get chunks in arbitrary sizes, at an arbitrary speed. |
22:26:58
| <dominictarr> | agree. |
22:27:02
| <isaacs> | 3. it cannot be assumed that a "end" event will follow immediately affter .end() |
22:27:10
| <dominictarr> | agree. |
22:27:35
| <dominictarr> | end may be buffered because the stream is paused. |
22:27:40
| <Raynos> | isaacs: https://github.com/Raynos/through-stream#example |
22:27:45
| <isaacs> | you even have half-duplex cases. |
22:27:53
| <Raynos> | what's half duplex |
22:28:10
| <isaacs> | half-duplex = we used to be duplex, but now one direction is closed |
22:28:12
| <Raynos> | a through-stream for me is a simple stream which has a shared buffer between the write & read methods |
22:28:22
| <dominictarr> | Raynos, it's when one side of the stream ends, but the other side can keep on writing |
22:28:37
| <isaacs> | dominictarr: well, with filters, that's actually the part you *don't* typically have. |
22:28:44
| <isaacs> | dominictarr: but you do have the revers |
22:28:50
| <isaacs> | (no longer writable, but can keep reading) |
22:29:24
| <Raynos> | isaacs: a through stream is a stream which when written to makes some or no manipulation to the underlying Buffer and when read from simply returns n bytes from the Buffer |
22:29:36
| <dominictarr> | isaacs, that couldn't have any effect on any thing. |
22:29:38
| <isaacs> | Raynos: zlib can't be safely done in place |
22:29:51
| <isaacs> | you need separate input and output buffers. |
22:29:59
| <Raynos> | isaacs: why? |
22:30:07
| <isaacs> | Raynos: consider a file that is 100GB of "A" |
22:30:12
| <Raynos> | oh yeah I see your point |
22:30:17
| <isaacs> | Raynos: now you compress that file. and it's now tiny |
22:30:19
| <Raynos> | you need a seperate window buffer |
22:30:24
| <isaacs> | now you take the compressed bit, and decompress it |
22:30:25
| <Raynos> | or a list of windows |
22:30:47
| <isaacs> | you write the puny 256 buffer or whatever, and zlib goes, "Ok, this says to emit 'A' for the next 100GB bytes. Get ready..." |
22:31:21
| <isaacs> | similar issues exist for crypto |
22:31:26
| <dominictarr> | isaacs, half-duplex streams are broke at the moment |
22:31:37
| <isaacs> | dominictarr: all streams are broke at the moment :) |
22:31:43
| <isaacs> | we just have a lot of duct tape everywhere :) |
22:31:55
| <isaacs> | wrapped around all our pipes |
22:31:58
| <isaacs> | or "ducts" if you will |
22:32:06
| <dominictarr> | yeah, one bit is this https://github.com/joyent/node/blob/master/lib/stream.js#L109 |
22:32:26
| <isaacs> | yeah |
22:32:40
| <isaacs> | luckily no one actually uses half-duplexes that muhc |
22:33:00
| <isaacs> | but we had to work around a bunch of bullshit in the zlib binding to not fall apart on that |
22:33:02
| <dominictarr> | I was just implementing something that needed that this morning actually! |
22:33:33
| <dominictarr> | but I decided to assume a telephone - where either side hangs up, but you say "goodbye" first. |
22:33:52
| <isaacs> | yep |
22:33:54
| <isaacs> | it's a pita |
22:33:54
| <dominictarr> | isaacs, anyway, through streams are easy. |
22:33:58
| <isaacs> | right |
22:34:02
| <dominictarr> | what is the problem with duplex streams. |
22:34:02
| <isaacs> | but not all filters are through streams. |
22:34:13
| <isaacs> | so, the issue is that we have all these fucking fields and magic bs things. |
22:34:31
| <isaacs> | water marks and "_flowing=true" and "_dests=[one,two,three]" |
22:34:42
| <isaacs> | and you have to be careful not to mess with any of htem. |
22:34:45
| <isaacs> | so, i'm thinking... |
22:34:50
| <dominictarr> | I'm surprised you put in multipiping. |
22:34:50
| <isaacs> | why not have a _readableOptions |
22:35:07
| <isaacs> | at least then it's only one thing to mess with |
22:35:09
| <dominictarr> | that could have just been a tee stream. |
22:35:40
| <isaacs> | dominictarr: yeah, but a bunch of folks pointed out real-world cases that would break without multipipe |
22:35:46
| <isaacs> | and i has a compassion for users. |
22:35:59
| <dominictarr> | isaacs, link? |
22:36:16
| <dominictarr> | I mean bash uses tee, whats the problem? |
22:37:30
| <isaacs> | dominictarr: i'd rather not push a change that's going to be a pita for voxer or joyent. |
22:37:44
| <dominictarr> | right. |
22:38:09
| <isaacs> | dominictarr: mostly it's logging stuff |
22:38:22
| <isaacs> | but i think we can get most use cases covered even better with a progress event of some sort |
22:38:34
| <isaacs> | or maybe, if you have something piping, we can emit "data" events |
22:38:36
| <dominictarr> | maybe a have a monkey patch that cleverly installs itself. |
22:38:51
| <isaacs> | since, at that point, the draw will make sure you get all the way through it eventually |
22:39:12
| <isaacs> | so, the issue with water marks, is that i'd also like them for writable streams |
22:39:29
| <isaacs> | and then there was this collision of key names. |
22:39:35
| <isaacs> | and also buffer and length and so on |
22:39:43
| <dominictarr> | so you know how much you have filled the buffer? |
22:39:49
| <isaacs> | sure |
22:40:48
| <dominictarr> | then you could do a lazy pipe, because you could see whether the stream is already "paused" |
22:41:23
| <isaacs> | yeah |
22:41:39
| <isaacs> | of course, now we have three "pipe" implementations: |
22:41:43
| <isaacs> | 1. the original old legacy one |
22:41:52
| <isaacs> | 2. the new write() while read() one |
22:41:54
| <Raynos> | dominictarr: What's a tee stream? |
22:42:01
| <dominictarr> | Raynos, man tee |
22:42:06
| <isaacs> | 3. the monkey-patched pipe() after already piping one |
22:42:22
| <isaacs> | Raynos: think actual plumbing pipes |
22:42:24
| <Raynos> | I find multipiping important |
22:42:28
| <isaacs> | one in, multiple out |
22:42:38
| <dominictarr> | well, the legacy pipe was already good at multipiping |
22:42:40
| <Raynos> | otherwise I need some kind of split stream |
22:42:52
| <isaacs> | dominictarr: yeah, but it sucked at everythign else :) |
22:42:54
| <Raynos> | and then I would end up always using the split stream |
22:43:26
| <dominictarr> | isaacs, a lot of people complained about pause() |
22:43:45
| <dominictarr> | you mean that? |
22:44:05
| <isaacs> | dominictarr: well, more to the point, the existing readable stream interface sucks at eerythign else |
22:44:12
| <isaacs> | dominictarr: pause doesn't pause, etc. |
22:44:38
| <dominictarr> | maybe it's just that I got good at streams, but I really didn't have a problem with the old interface. |
22:44:53
| <isaacs> | dominictarr: i'm pretty good at streams, but for lowlevel stuff, they really kind of suck |
22:44:59
| <isaacs> | lots of edges. |
22:45:33
| <Raynos> | dominictarr: I started writing the new readable streams, they are easy to write. |
22:45:49
| <dominictarr> | yeah, that is why I spent so much time looking for common abstractions to make implementing streams easier. |
22:45:52
| <Raynos> | I havn't had the need to create instances of PauseStream and BufferStream everywhere anymore |
22:46:11
| <dominictarr> | like, take an async function, make it into a stream. |
22:46:15
| <Raynos> | nor have I had the need to write duplex streams recently |
22:49:20
| <dominictarr> | isaacs, streams have too many states. that makes them hard. maybe if we can reduce the number of states...? |
22:50:17
| <Raynos> | dominictarr: I've reduced complexity by having only readable & writable streams and then wrapping the readable or writable stream with the filter. |
22:50:31
| <Raynos> | Then there is only one input, one output and one pipe. Anything else is a transformation |
22:50:40
| <dominictarr> | Raynos, yeah. that isn't good enough for core though. |
22:50:45
| <dominictarr> | that adds overhead. |
22:51:18
| <Raynos> | I doubt it does, you can benchmark it against pipes |
22:51:42
| <Raynos> | net.createServer(function (sock) { new Gzip(sock).pipe(sock) }) |
22:52:01
| <Raynos> | monkey patching read is cheaper then having two pipies |
22:52:04
| <Raynos> | pipes* |
22:52:22
| <dominictarr> | Raynos, I've benched stuff, like passing through extra naked through streams makes a difference. |
22:52:50
| <dominictarr> | that you can notice, if you are doing heavy IO. |
22:53:18
| <isaacs> | dominictarr: so, "given a read(n, cb)" make a readable stream |
22:53:26
| <isaacs> | dominictarr: that's effectively what Readable is |
22:53:37
| <dominictarr> | that should do it for fs. |
22:53:44
| <isaacs> | dominictarr: and for tcp |
22:53:54
| <dominictarr> | I implemented https://github.com/dominictarr/fs-reverse/blob/master/index.js like that. |
22:54:03
| <isaacs> | dominictarr: remember, on windows, there's no sync read() on tcp streams |
22:54:15
| <isaacs> | and, you need to space out the actual syscalls |
22:54:28
| <isaacs> | so _read(n, cb) is only ever called with 16*1024 or something |
22:54:38
| <isaacs> | and then read(n) is how you actually consume that buffer. |
22:55:08
| <isaacs> | and when it gets below some level, you do another _read(n, cb) |
22:56:00
| <dominictarr> | that makes sense. |
22:56:35
| <dominictarr> | no way to know where the level should be though. |
22:57:20
| <dominictarr> | the hard thing about streams is testing them. I'd want to produce elegant reference implementations, and then ugly performant implementations. |
22:57:44
| <substack> | awesome that taiwan happens to have the same kind of electrical outlet as north america |
22:57:57
| <substack> | seeing what shanghai uses |
22:58:37
| <dominictarr> | substack, in cambodia/thailand they where actually multiadapters and nz/aus plugs ALSO WORK |
22:58:46
| <dominictarr> | I think euro works too. |
22:59:21
| <dominictarr> | just not UK/ireland... the world's ugliest adapters. |
22:59:52
| <substack> | oh goodness it looks like china uses all the possible adapters |
22:59:54
| <substack> | at once |
23:00:09
| <dominictarr> | china ++ |
23:00:21
| <substack> | http://www.travelchinaguide.com/essential/electricity.htm |
23:00:25
| <dominictarr> | well, they manufacture them all. |
23:00:37
| <substack> | true |
23:01:11
| <substack> | this is crazy: http://www.madaboutshanghai.com/2005/11/power_outlets_i.html |
23:02:16
| * tphummel | joined |
23:02:31
| <substack> | (crazy excellent) |
23:02:32
| <dominictarr> | isaacs, watermarks add an extra layer between read() and and actual syscall. |
23:02:47
| <dominictarr> | are you talking about anticipating the next call? |
23:02:59
| <isaacs> | dominictarr: yes, watermarks add an extra layer betwee read() and an actual syscall |
23:03:02
| <isaacs> | dominictarr: that's by design |
23:03:05
| <substack> | china is like a gigantic kludge of a distributed system |
23:03:09
| <substack> | in the best possible way |
23:03:14
| <isaacs> | dominictarr: you don't want to be calling read(2) for 4 bytes. |
23:03:19
| <isaacs> | that's very expensive. |
23:03:53
| <dominictarr> | substack, thailand/cambodia has just the top half of that socket in the photo - but I still managed to jam a NZ plug in. |
23:04:18
| <isaacs> | dominictarr: so yes, you anticipate the syscall |
23:04:44
| <isaacs> | when you consume down to only 1024 bytes in the buffer, it fetches another 16kb |
23:04:57
| <dominictarr> | just to be the devil's advocate, if you only made the syscall when the buffer was empty, how much simpler would that make the implementation? |
23:05:29
| <substack> | china knows all about concurrent versioning |
23:05:51
| <substack> | they've probably been using a package manager like npm for thousands of years |
23:06:26
| <dominictarr> | substack, have you read "art of war" and / or "tao de ching" |
23:06:29
| <dominictarr> | ? |
23:06:42
| <isaacs> | dominictarr: it'd be slower |
23:06:46
| <isaacs> | dominictarr: and not much simpler. |
23:06:51
| <substack> | nope |
23:07:08
| <isaacs> | dominictarr: so, you have something like: veryFastReader.pipe(verySlowWriter) |
23:07:25
| <isaacs> | dominictarr: or the reverse: verySlowReader.pipe(veryFastWriter) |
23:07:30
| <dominictarr> | isaacs, like fs -> network |
23:07:34
| <isaacs> | yeah |
23:07:46
| <isaacs> | or, like local network -> cell phone network |
23:07:50
| <isaacs> | much bigger diff |
23:07:51
| <dominictarr> | slow -> fast means lots of buffering is good. |
23:08:00
| <dominictarr> | yup |
23:08:29
| <substack> | blarg everybody speaks english here |
23:08:32
| <isaacs> | the worst is when you have shit where it says, "That write couldn't flush" so you pause the reader, then it does flush right away, and emits drain |
23:08:44
| <substack> | silly rich country |
23:08:51
| <dominictarr> | I'm guessing that fast -> slow would not make so much difference. |
23:09:01
| <isaacs> | substack: traveling the world for conferences has taught me that other countries are populated mostly by english speaking computer nerds |
23:09:17
| <dominictarr> | substack, still, if you know hello and thank you that wins you points. |
23:09:30
| <substack> | I have 2 pages filled with phrases |
23:09:40
| <substack> | haven't had a chance to use them yet! |
23:09:52
| <dominictarr> | TODO: GOTO PLACE THAT DOES NOT SPEAK ENGLISH. MAYBE FRANCE. |
23:09:53
| <LOUDBOT> | I THINK MY PIE IS GOING TO BE WATERY |
23:10:22
| <substack> | france actually can speak english, they just militantly choose not to |
23:10:38
| <dominictarr> | I respect them for that. |
23:10:44
| <substack> | same |
23:11:33
| <dominictarr> | you don't even need to speak at all for most important things, eating, finding a place to sleep, etc. |
23:11:57
| <dominictarr> | 99% of what people say is completely unnecessary. |
23:11:59
| <Raynos> | dominictarr: I'm giving a talk at LNUG. Thinking about doing "reactive programming with streams". Any better suggestions? |
23:12:25
| <dominictarr> | Raynos, that is good. I'm gonna do "writing custom streams" |
23:12:50
| * substack | doing a survey of streaming libs at lisbon |
23:13:18
| <st_luke> | crap I have to buy a plane ticket |
23:13:23
| <st_luke> | that shit expensive |
23:13:28
| <dominictarr> | hitch hike. |
23:13:38
| <dominictarr> | substack, what about dublin? |
23:13:41
| <st_luke> | on a boat |
23:13:50
| <dominictarr> | st_luke, where do you live? |
23:13:55
| <st_luke> | dominictarr: nyc |
23:14:02
| <substack> | dominictarr: something more similar to what I'm doing for shanghai about how littleness is good |
23:14:14
| <dominictarr> | it's usually cheaper if you brought the ticket yesterday. |
23:14:15
| <substack> | except with leprechauns instead of dragons |
23:14:26
| <dominictarr> | haha. |
23:14:37
| <dominictarr> | that is very tao de ching |
23:15:25
| <dominictarr> | {ruling a country, writing a module} is like flying a small fish. |
23:15:39
| <dominictarr> | (don't poke it too much) |
23:16:08
| <substack> | also I found this: http://zh.wikipedia.org/zh-cn/Unix%E5%93%B2%E5%AD%A6 |
23:17:50
| * substack | listening to 90s alt country |
23:18:05
| <dominictarr> | heh, it links to "elements of style" a book I brought, but never got around to reading. |
23:18:06
| <substack> | most unfitting music possible |
23:18:13
| <substack> | for this venue |
23:18:24
| <dominictarr> | it's not about unix, I'm pretty sure. |
23:18:37
| <substack> | yeah |
23:18:51
| <substack> | it's just that the people building unix stumbled upon this design philosophy that works |
23:22:56
| <dominictarr> | people who design things always have very wide interests. |
23:23:18
| <substack> | I said this on twitter but it's so true |
23:23:18
| <dominictarr> | they turn over every rock, searching for answers. |
23:23:24
| <substack> | bell labs was the bauhaus of computing |
23:23:45
| <substack> | a bunch of smart people bouncing ideas off each other for a brief time and getting to the essence of design |
23:23:58
| <substack> | yes, thoroughly exploring the solution space |
23:24:07
| <substack> | not getting stuck on local maxima |
23:24:22
| <st_luke> | substack: sometimes I wish I could live through that era |
23:24:35
| <st_luke> | if I had a holodeck I would spend a lot of time at bell labs |
23:24:36
| <substack> | st_luke: well then the nazis came and they all fled to america |
23:24:44
| <substack> | oh hah bell labs |
23:25:41
| <st_luke> | is mbalho active? |
23:25:42
| <substack> | st_luke: just watch that at&t archive footage on a loop |
23:26:14
| <substack> | it's amazing that a company as thoroughly evil as at&t could create something as beautiful and benevolent as unix |
23:26:25
| <isaacs> | dominictarr: large swaths of south america have very few english speakers. |
23:26:36
| <substack> | maybe extreme evil seeks extreme goodness or some such platitude |
23:27:00
| <isaacs> | substack: i think they actually got eviler when the govt forced them to split up |
23:27:09
| <st_luke> | substack: sometimes in these giant companies with too much money people gather and realize they can use the resources to make something special |
23:27:12
| <st_luke> | never lasts long though |
23:27:13
| <substack> | the evil got concentrated, perhaps |
23:27:25
| <substack> | like terrorist splinter cells |
23:27:26
| <dominictarr> | substack, st_luke but isn't oakland that right now? |
23:27:46
| <isaacs> | ok, i'm gonna pull the high/low watermark from the Readable stream |
23:27:54
| <st_luke> | dominictarr: from my experience in oakland (which is fairly limited), I would say it feels like that sort of place |
23:27:58
| <isaacs> | fs and net streams will just bolt that bit on |
23:28:38
| <dominictarr> | isaacs, ++ |
23:29:03
| <st_luke> | if I were to move somewhere in the US after I'm done in new york it would probably be oakland |
23:29:47
| <dominictarr> | "after I'm done in new york"? I've never been to nyc, but listening to frank sinatra, how is that possible? |
23:30:18
| <substack> | the northeast is my only remaining US blackspot |
23:30:31
| <isaacs> | st_luke: yes. |
23:30:41
| <isaacs> | st_luke: oakland is the correct place to end up |
23:31:08
| <isaacs> | st_luke: after new york, sf might be an easier transition, though |
23:31:12
| <st_luke> | substack: the northeast is a nice place for the geography |
23:31:14
| <isaacs> | unless you find yourself fed up with cities in general. |
23:31:19
| <dominictarr> | there is also I think an important factor; freedom. I've always done my best work when the boss was on holiday. |
23:31:25
| <substack> | been to 43/50 of the states |
23:31:34
| <isaacs> | but like, if you miss the smell of pee and the oppressive weight of concrete, oakland might be a shock to the system. |
23:31:50
| <dominictarr> | I heard that when they where writing unix, they where _meant_ to be writing a word processor. |
23:31:52
| <st_luke> | isaacs: that pee smell never gets old |
23:32:24
| <isaacs> | st_luke: i've noticed that the thing that the mission and the bronx have in common is that people who *live* there claim "it's not that bad" |
23:32:24
| <substack> | dominictarr: that might explain the spell checker that kernighan busts out in that 1982 video |
23:32:29
| <isaacs> | and people who visit gag constantly. |
23:33:05
| <st_luke> | isaacs: i was really disappointed with the mission, I had the wrong idea of what it would be |
23:33:19
| <isaacs> | st_luke: yeah, it's probably the most overhyped place i've ever been. |
23:33:24
| <dominictarr> | substack, yeah. and maybe the whole "streams of text" thing too. |
23:33:33
| <isaacs> | the thing that's good about it is that there are a lot of people socializing all the time. |
23:33:37
| <isaacs> | many bars and shit. |
23:33:45
| <isaacs> | (not "and stuff", i mean literal feces) |
23:34:07
| <dominictarr> | once, I googled "hipster bars" and went out in the mission, but I had way more adventures coming back to oakland on the BART |
23:34:22
| <st_luke> | I live in a neighborhood with a whole lot of bars and hipsterish people right now |
23:34:31
| <st_luke> | that's more or less what people in brooklyn do |
23:35:35
| <substack> | I hardly ever go to bars |
23:35:47
| <substack> | only when I know some people who will be there |
23:36:04
| <dominictarr> | go to a bar, look around, is anyone having fun? no. go home. |
23:36:19
| <substack> | seems like a good algorithm |
23:36:23
| <dominictarr> | most people have no idea why they go to bars. |
23:36:56
| <substack> | I find it's much easier to strike up conversations at coffee shops |
23:37:02
| <substack> | since people tend to be working on interesting things |
23:37:16
| <substack> | and it's quiet enough to talk |
23:37:16
| <dominictarr> | and not so loud. |
23:37:23
| <st_luke> | the thing that looks cool about the bay area and oakland is people doing tech meetups for the sake of just getting computers out and playing around on them |
23:37:31
| <st_luke> | maybe building something, in a group or individually |
23:37:35
| <substack> | like once at awaken I saw a guy working on some category theory stuff |
23:37:54
| <st_luke> | tech meetups in nyc are about the networking aspect more than anything, not people sitting around discovering things that their computer can do |
23:38:01
| <substack> | and my passing familiarity with it via haskell was enough to have some back and forth |
23:38:29
| <st_luke> | substack: that's a cool experience |
23:38:40
| <substack> | berkeley is really great for that stuff |
23:38:44
| <substack> | with all the students around |
23:39:00
| <st_luke> | reminds me of the story I read about the guy who ran into someone at a coffee shop who invented the computer |
23:39:12
| <substack> | arguing about economics or sociology experiments |
23:39:19
| <st_luke> | yeah |
23:39:38
| <st_luke> | it's why I like to go to hack events and not participate in the competition part |
23:39:44
| <substack> | oakland has more of the artist/radical vibe which is neat too |
23:40:08
| <substack> | I see folks like boots riley just walking around down by the plaza |
23:40:34
| <substack> | and then protests shut down downtown once every few months |
23:42:16
| <st_luke> | yeah, I think I might end up there sometime |
23:42:24
| <st_luke> | but I would like to try a few months in Europe first |
23:42:30
| <substack> | rad |
23:42:32
| <dominictarr> | oh, man. he has an album named "genocide & juice" |
23:42:34
| <substack> | crazy adventures |
23:43:38
| <st_luke> | yeah, maybe Berlin and Russia if I can save up some money |
23:45:04
| <isaacs> | substack: the loudness of bars is intentional. |
23:45:26
| <substack> | isaacs: so boring people don't get discovered? |
23:45:37
| <isaacs> | substack: the purpose is to make you have to lean in, and to increase the difficulty of speaking to the point where we detach our analytical minds and start speaking mostly lower-level primate |
23:45:55
| <substack> | I hate everything about that. >_< |
23:46:00
| <isaacs> | noise -> physical closeness, lack of linguistic precision |
23:46:07
| <isaacs> | couple with alcohol, and the effect increases. |
23:46:15
| <dominictarr> | just stop talking and use body language. |
23:46:19
| <isaacs> | exactly |
23:46:21
| <isaacs> | or, learn ASL |
23:46:33
| <dominictarr> | charades is more fun. |
23:46:42
| <isaacs> | going to bars with marisa is great, because she's *always* hard of hearing, read lips really well, and we can sign to each other. |
23:46:45
| <st_luke> | I like when bars get closed down and are used for small conferences |
23:46:47
| <isaacs> | it's like having a secret language. |
23:47:17
| <dominictarr> | srsly, charades is great fun. but be warned, you start to invent your own language. |
23:47:28
| <substack> | st_luke: especially when they turn the music way down |
23:48:07
| <st_luke> | yeah |
23:48:54
| <st_luke> | actually I think outside of conferences I hardly ever drink |
23:50:02
| <st_luke> | hopefully synthehol will be invented soon |
23:50:10
| <st_luke> | or marketed |
23:52:42
| <dominictarr> | I heard there is a pill that near-instantly makes you sober |
23:53:22
| <dominictarr> | but it was too much of a legal minefield, you'd still be legally drunk due to blood alcohol. |
23:53:57
| <substack> | dominictarr: oh there was a race condition that somebody fixed recently in browserify |
23:54:03
| <substack> | could be related to that bug you just closed |
23:54:05
| <dominictarr> | I saw that. |
23:54:18
| * substack | is terrible about closing issues |
23:54:42
| <dominictarr> | you are spread too thin, too many modules. |
23:54:49
| <dominictarr> | you need to delegate. |
23:55:41
| <substack> | it's only a handful of modules that give a ton of noise |
23:56:00
| <substack> | browserify is super noisy |
23:57:15
| <isaacs> | substack: i hear ya |
23:57:31
| <isaacs> | substack: every so often i take a saturday to close issues in npm |
23:57:37
| <isaacs> | usually there's 10 or 20 dupes of every actual issue |
23:57:46
| <isaacs> | or things that people figured out, but never came back to resolve |
23:58:01
| <st_luke> | those are the worst |
23:58:02
| <isaacs> | nominating helpers is winful |
23:58:20
| <substack> | I have >100 modules that pretty much have no noise at all |
23:58:31
| <substack> | those tend to be the very tiny hyper-focused modules with lots of tests |
23:58:50
| <st_luke> | sometimes I don't like opening issues for what I think might be silly things, but other times I like opening issues on modules released by companies that look like they do it partially for marketing reasons |
23:58:53
| <substack> | best kind of module |
23:59:10
| <substack> | people who complain about modules being too small do not have this problem for the most part |