00:53:32  <rowbit>/!\ ATTENTION: (default-local) [email protected] successfully signed up for developer browserling plan ($20). Cash money! /!\
00:53:32  <rowbit>/!\ ATTENTION: (default-local) paid account successfully upgraded /!\
00:54:06  <substack>yay
00:54:10  <substack>cash dolores
00:54:33  * _sorensenquit (Quit: _sorensen)
02:10:14  * tphummelquit (Quit: tphummel)
02:32:33  * wiwilliajoined
03:06:02  * wiwilliaquit (Ping timeout: 244 seconds)
03:17:21  <substack>http://substack.net/images/keeping_it_surrealtime.svg
03:37:49  <niftylettuce>Please upboat on HN "Show HN: Chrome Extension to automatically block the annoying Quora login prompt"
03:38:09  <niftylettuce>substack chapel pkrumins paul_irish dominictarr dools AvianFlu rook2pawn tanepiper ryanseddon
03:38:13  <niftylettuce>:D QUORA POOOOP
03:38:13  <LOUDBOT>FOX NEWS: OBAMA LEGISLATES MANDATORY ABORTION, HOMOSEXUALITY; DECLARES BURNING BIBLES AS A CORE COMPONENT OF NEW ENERGY PLATFORM
03:38:55  <AvianFlu>LOUDBOT: twitlast
03:38:56  <LOUDBOT>AvianFlu: http://twitter.com/loudbot/status/245365534487629824 (adipocere/#mefi)
03:41:05  * doolslikes his lettuce nifty
03:41:25  <niftylettuce>GIGGITy
03:41:26  <LOUDBOT>LET'S IMAGINE THIS: YOU HAVE TWO PEOPLE THAT ARE BOTH EATING A HOT DOG IN NY
03:42:21  <substack>the niftiest lettuce
03:42:37  <substack>I'm going to keep hurling art at people until everything looks awesome
03:47:56  <niftylettuce>:D :D
03:48:01  <niftylettuce>substack: :) <3
03:54:41  <niftylettuce>thx all
03:55:18  <chapel>http://skeu.it/post/25597190946/via-alf-where-it-all-began-via-perezd-its
04:10:07  * st_luke_joined
04:16:28  * st_lukequit (Disconnected by services)
04:16:30  * st_luke_changed nick to st_luke
04:16:58  * st_luke_joined
04:28:49  <substack>getting some hella chinese leveling up
04:29:03  <substack>before heading out
04:31:08  <AvianFlu>substack: ARE YOU ALREADY IN THE BIG DATA CHINA MARKET!?
04:31:19  <AvianFlu>damn, I spoilt my own loudbot fun
04:31:21  <substack>YES INDEED
04:31:22  <LOUDBOT>GET THE FUCK OUT OF MY WAY BEFORE I BEAT YOU SENSELESS WITH MY FUCKING NUTS
04:31:24  <st_luke>CHINAMARKET
04:31:25  <LOUDBOT>WHICH IS A KNOWN ISSUE WITH CRAZY PEOPLE
04:31:31  <substack>是的,我
04:31:41  <AvianFlu>WTF LOUDBOT THOSE WERE TOTALLY CAPS
04:31:42  <LOUDBOT>LOL THAT'S A TERRIBLE IDEA RETARD
04:31:52  <st_luke>LMBO
04:32:07  <substack>我准备到大的数据对整个中国市场
04:32:23  <dools>CHINESE CHARACTERS MAKE MY TERMINAL GO FUNNY. IT'S A CONSPIRACY
04:32:24  <LOUDBOT>ACHIEVEMENT UNLOCKED: ACHIEVEMENT CERTIFICATE
04:33:37  <substack>AND AWAY I GO BITCHES
04:33:38  <LOUDBOT>I WAS TYPING STARTX WHEN CLINTON WAS PRESIDENT, BITCH.
04:34:12  <AvianFlu>LOUDBOT: twitlast
04:34:12  <LOUDBOT>AvianFlu: http://twitter.com/loudbot/status/245379444481601536 (Dav3xor/#dongtown)
04:43:18  * st_luke_quit (Disconnected by services)
04:43:47  * st_luke_joined
04:52:57  <st_luke>got really excited about seeing a libdb driver in npm
04:53:04  <st_luke>but it was just for libdbus
05:00:49  <st_luke>hooray https://github.com/mcavage/node-bdb
05:01:46  <guybrush>seems like not maintained and no gyp
05:02:47  <st_luke>yep
05:03:02  <st_luke>barely any js in it anyway
05:11:55  * mikealquit (Quit: Leaving.)
05:12:45  * mikealjoined
05:45:32  * st_lukequit (Remote host closed the connection)
05:46:36  * dominictarrquit (Ping timeout: 240 seconds)
06:25:38  <guybrush>sweeeeeeeeeet https://github.com/mranney/node_redis/blob/master/package.json#L2
06:25:48  <guybrush>with superfast js-parser
06:39:03  * st_lukejoined
06:54:13  * idletomquit (Ping timeout: 268 seconds)
07:01:28  * substackat sfo
07:06:04  * st_lukequit (Remote host closed the connection)
07:14:02  * tilgoviquit (Remote host closed the connection)
07:51:19  <rowbit>/!\ ATTENTION: (default-local) paid account successfully upgraded /!\
07:51:20  <rowbit>/!\ ATTENTION: (default-local) [email protected] successfully signed up for developer browserling plan ($20). Cash money! /!\
07:53:05  <guybrush>substack: what do you think about making browserify().require('child_process').spawn() creating a webworker?
07:54:17  <guybrush>oh maybe this is wrong when i think more about it
07:56:18  <guybrush>send and .on('message') could be mapped to postMessage and worker.on('message'
07:56:59  <guybrush>but everything else would behave very different :p
07:57:29  <substack>yay
07:57:58  <guybrush>btw webkitPostMessage is so fucking sexy
07:58:37  <substack>sweets
07:58:45  <substack>guybrush: make might sense to add that to child_process
07:59:01  <substack>although it reads from files
07:59:04  <substack>so that's a bit tricky
07:59:15  <substack>can always do a separate module for it
07:59:23  <guybrush>yes
07:59:27  * substackboarding very soon
07:59:37  <guybrush>oh have fun!
07:59:58  <guybrush>if there are some slides, i cant wait to see them :p
08:30:58  <AvianFlu>LOFTY PRIORITIES INDEED
08:30:59  <LOUDBOT>I'M FLYING TO BOSTON TODAY MAYBE I WON'T DIE EVEN
08:47:22  * ITproquit (Read error: Connection reset by peer)
08:48:27  * ITprojoined
09:15:16  * ITproquit (Ping timeout: 260 seconds)
09:20:58  * AvianFluquit (Quit: AvianFlu)
12:31:33  * timoxleyjoined
13:04:55  * st_lukejoined
13:15:08  * idletomjoined
13:16:21  * dominictarrjoined
13:32:32  * idletomquit (Ping timeout: 248 seconds)
13:34:42  * st_lukequit (Remote host closed the connection)
14:19:24  * st_luke_changed nick to st_luke
14:38:03  * wiwilliajoined
14:43:08  * dominictarrquit (Quit: Leaving)
14:43:24  * dominictarrjoined
14:46:06  * timoxleyquit (Quit: Computer has gone to sleep.)
15:03:04  * AvianFlujoined
15:08:39  * _sorensenjoined
15:13:06  * timoxleyjoined
15:20:45  * juliangruberjoined
15:31:33  * _sorensenquit (Remote host closed the connection)
15:41:49  * juliangruberquit (Ping timeout: 245 seconds)
15:46:24  <dominictarr>Raynos, new flights booked! coming to london!
15:47:46  <Raynos>sweet
15:48:09  <Raynos>dominictarr: If you land during the day I may come meet you in london
15:49:08  <dominictarr>Raynos, I land 18:30 LHR
15:50:48  <Raynos>cant make that
16:18:20  * wiwilliaquit
16:45:44  * dominictarrquit (Ping timeout: 276 seconds)
17:02:15  * mikealquit (Quit: Leaving.)
17:02:46  * mikealjoined
17:36:11  * idletomjoined
18:17:38  * _sorensenjoined
18:18:32  * mikealquit (Quit: Leaving.)
18:39:52  * dominictarrjoined
18:58:33  * py1honquit (*.net *.split)
18:59:43  * py1honjoined
19:07:34  * mikealjoined
19:14:23  * tilgovijoined
19:36:02  * idletomquit (Ping timeout: 246 seconds)
19:47:39  * simcop2387quit (Excess Flood)
19:48:39  * simcop2387joined
19:48:54  * mikealquit (Quit: Leaving.)
19:58:12  * dominictarrquit (Remote host closed the connection)
20:22:01  * mikealjoined
21:09:23  * ryan_stevensjoined
21:09:38  * tilgoviquit (Ping timeout: 276 seconds)
21:21:05  <mbalho>if anyone wants a streaming challenge fork this and make it purely stream https://github.com/maxogden/stenographer
21:21:49  <mbalho>right now it buffers the entire input audio stream https://github.com/maxogden/stenographer/blob/master/index.js#L30
21:40:21  * tilgovijoined
21:51:07  * substackin taipei
21:51:30  <dools>you've got a real taipei personality
21:52:46  <substack>seems pretty japanesque
21:52:52  * dominictarrjoined
21:52:58  <substack>dominictarr: \o
21:53:03  * substackin taipei
21:54:03  <dominictarr>wow, sweet.
21:54:22  <dominictarr>substack, how is the street-food?
21:54:49  <substack>just at the airport
21:54:58  <substack>and everything is closed since it's early
21:57:30  <substack>and I don't have any NT$
21:58:32  <substack>29.47NT$ = 1 USD
21:58:41  <substack>so stuff is pretty cheap
22:00:03  <Raynos>substack: Does marak still liev with you?
22:00:20  <substack>yes
22:00:58  <dominictarr>hes going to india soon. needs a holiday.
22:01:33  <dominictarr>substack, I'd bet they'd take USD, near the airport.
22:02:04  <substack>if they have credit card readers those do the conversion automatically
22:02:33  <dominictarr>in Cambodia, there would be streetfood pretty much any time at all.
22:02:54  <dominictarr>maybe there would be 4 hours when there is no street food.
22:03:32  <dominictarr>but in the middle of the night, there would be kids walking around banging on this wood block thing.
22:04:01  <dominictarr>first I thought they where just into music -- some of them where pretty funky -- but turned out they where selling noodles.
22:06:11  <substack>some shops look open now
22:06:24  <substack>getting some espresso I think
22:06:41  <substack>actually I'll wait until my laptop is fully charged, just 30 more minutes
22:07:05  <dominictarr>substack, you are waiting for a connecting flight?
22:07:05  <substack>I intend to get lost in shanghai later today
22:07:07  <substack>yep
22:07:25  <dominictarr>substack, get lost ++
22:07:45  <dominictarr>maybe rent a bicycle
22:07:53  <substack>good plan
22:08:01  <dominictarr>how long are you in shanghai for?
22:08:53  <substack>I leave on the 17th
22:09:00  <substack>5 days
22:09:05  <st_luke>awesome
22:09:36  <st_luke>substack: what airline did you fly out on?
22:09:42  <substack>china airlines
22:10:16  <st_luke>foreign airlines are fu
22:10:17  <dominictarr>find a friendly looking tuktuk driver, and take his number, and make him your tuktuk driver for your stay.
22:10:18  <st_luke>fun
22:10:39  <dominictarr>thats what I will do next time I'm in Asia.
22:10:54  <st_luke>dominictarr: good idea
22:11:17  <st_luke>when I was in thailand I used the same motorcycle taxi driver a bunch of times and it made it way more fun
22:11:24  <dominictarr>he'll take you to eat at all his relatives and friends resturants, etc.
22:11:38  <isaacs>dominictarr, Raynos: So, i'm bumping into a problem with some of the things we're talking about with this readable stream api
22:11:45  <isaacs>the problem is that there's too many things now
22:11:51  <Raynos>isaacs: :D
22:11:54  <dominictarr>haha
22:12:07  <isaacs>i'm considering scrapping the low/hight water marks
22:12:14  <Raynos>I don't use them myself
22:12:22  <isaacs>Raynos: they're important for real life things
22:12:29  <dominictarr>isaacs, that stuff should be in userland.
22:12:30  <isaacs>you don't use them because yoer' dealing with synthetic streams almost exclusively
22:12:36  <isaacs>dominictarr: well, tcp and fs is where you need it most.
22:12:39  <isaacs>(ie, at all)(
22:12:46  <dominictarr>you just want a really really simple base class.
22:13:08  <isaacs>i want a class that will work for all the things in node, also
22:13:09  <Raynos>isaacs: https://gist.github.com/c9d21023a2e62ebdaf54
22:13:23  <dominictarr>and then let subclasses inherit from it, or use intermediates that make sense for their particular cas.
22:13:24  <dominictarr>e
22:13:26  <Raynos>thats my implementation of multi party pipe if it helps
22:13:53  <dominictarr>there are a bunch of types of streams, and each has slightly different reqs.
22:14:04  <isaacs>Raynos: y reemit(stream, target, ["end"])
22:14:08  <isaacs>Raynos: what is that? ^
22:14:18  <isaacs>dominictarr: tcp and fs are all i really care about.
22:14:33  <dominictarr>isaacs, those are quite different types.
22:14:40  <isaacs>dominictarr: yes
22:14:45  <dominictarr>fs is really simple. just read and write.
22:14:46  <isaacs>dominictarr: but that covers basically all the possible use cases.
22:14:53  <isaacs>dominictarr: well, fs is pretty complicated actually
22:15:05  <dominictarr>no no no, duplex is the complexist
22:15:15  <isaacs>because reads are necessarily async, and the output can never actually be flushed, but you ought to pretend it is flushed if ther's only one buffered.
22:15:24  <dominictarr>then filter streams are different
22:15:26  <isaacs>but yes, duplex is the complexist, for sure
22:15:33  <dominictarr>zlib != duplex.
22:15:35  <isaacs>filters are just a special case of duplex
22:16:00  <isaacs>proof: net.createServer(function (sock) { sock.pipe(new Gzip()).pipe(sock) })
22:16:04  <Raynos>isaacs: thats a shitty implementation of `source.on("end", function () { target.end() })`
22:16:11  <dominictarr>isaacs, that is what I thought at first. but duplex is like a telephone, and filter is like a meat grinder.
22:16:27  <isaacs>dominictarr: in this case, then, a telephone can grind meat, also
22:16:37  <isaacs>whatever you need to do for duplex, it turns out you need to do that for filters as well.
22:16:50  <isaacs>a filter is: "Based on xyz input, produce abc output"
22:17:00  <isaacs>a duplex is: "you get output and you put in input"
22:17:17  <isaacs>the only diff is the causal connection, but a duplex stream might have that causal connection as well.
22:17:22  <dominictarr>that is the idea of a meat-grinder. you are transforming the input.
22:17:26  <isaacs>right
22:17:31  <Raynos>isaacs: filter is far easier then duplex
22:17:36  <dominictarr>no, duplex is communication with a black box.
22:17:38  <isaacs>but you could be transforming 10lb of meat into 50lb of motor oil
22:17:44  <dominictarr>you can send data to me...
22:17:51  <Raynos>with duplex you need to handlers, one to handle input and do something with it. And one to watch on some other thing that generates output and then generate output
22:17:55  <isaacs>gzipped output is not anything at all "like" its source
22:17:58  <dominictarr>but I am crazy. I could say ANYTHING in response.
22:18:05  <Raynos>with filter you have one handler, one that watches input and does something sync / async and then generates output
22:18:07  <dominictarr>or nothing.
22:18:19  <isaacs>again: filter is a special case of duplex.
22:18:24  <isaacs>i'm not saying filter = duplex
22:18:31  <isaacs>filter : duplex :: square : rectangle
22:18:33  <Raynos>The specific thing about duplex is that it's basically two streams, a readable & writable joined together. Whether a filter is a single readable / writable stream
22:18:48  <dominictarr>Raynos, well put.
22:18:56  <dominictarr>it's like two independent streams.
22:19:02  <isaacs>look, for any filter, you can have a duplex stream that is *exactly the same* as that filter stream
22:19:10  <Raynos>isaacs: but that's overly complex
22:19:18  <isaacs>is it?
22:19:26  <Raynos>any RW stream can always be a duplex stream of a R stream and a W stream
22:19:33  <Raynos>but a single RW stream is less complex then two streams
22:19:45  <isaacs>it turns out that whatever you need to do for duplex streams, filter streams are a subset of that, and it's not much less of a subset, really
22:19:56  <dominictarr>isaacs, the significant difference is that the pause behaviour is passed through from one side to the other in a filter.
22:19:56  <isaacs>the *only* thing you don't have to handle with filters is output coming before you write() anythign
22:20:11  <Raynos>the other thing is
22:20:20  <isaacs>dominictarr: not necessarily immediately.
22:20:42  <dominictarr>isaacs, no. not necessarily. but in most cases.
22:20:50  <isaacs>show me a difference other than "The output is causally connected (somehow, unspecifically, asynchronously) to the input"
22:21:00  <isaacs>"not necessarily but in most case" = "not a constraint"
22:21:01  <dominictarr>you could have a fliter stream that just changes buffers, for example.
22:21:22  <Raynos>isaacs: Does a filter stream imply filter1.pipe(filter2).pipe(filter1) is an infinite loop?
22:21:26  <isaacs>dominictarr: you could also have a filter stream that outputs either 2 or 20 depending on whether date.now() % 2 = 0, every third write.
22:21:36  <isaacs>Raynos: not necessarily
22:21:42  <isaacs>but in most cases :)
22:21:58  <isaacs>a tee is another kind of potential filter.
22:22:06  * idletomjoined
22:22:14  <isaacs>it has output, and input, and the output is causally connected (somehow) to the input (in that it's identical)
22:22:21  <Raynos>isaacs: a duplex stream does not suffer from duplex1.pipe(duplex2).pipe(duplex1) being an infinite loop
22:22:25  <isaacs>or, forget about tee, consider cat
22:22:33  <isaacs>Raynos: it might, though
22:22:41  <Raynos>only if they are echo streams
22:22:43  <isaacs>Raynos: depending on what the duplex is.
22:22:51  <isaacs>Raynos: orif they're just a conversation.
22:22:58  <isaacs>Raynos.pipe(Isaacs).pipe(Raynos)
22:22:58  <Raynos>and an echo stream is a stupid duplex stream that behaves like a through stream
22:23:05  <isaacs>that conversation could go on forever without ever repeating itself.
22:23:13  <dominictarr>isaacs, there is definately a fuzzy line between duplex and filter. but I have found it simpler to build one base class for through streams,
22:23:13  <isaacs>but you and i are not mere filters!!
22:23:17  <dominictarr>and another for duplex.
22:23:35  <isaacs>dominictarr: right, but what i'm saying is, from the pov of the interface, supporting one means you have to support the other.
22:23:38  <Raynos>isaacs: I agree with dominic that base classes for through, write, read & duplex reduce complexity
22:24:05  <isaacs>you cannot safely assume that you can just slap a filter(chunk) function that returns the transformed value (or even emits a transformed value later) and call it a day
22:24:10  <isaacs>zlib is the canonical filter stream.
22:24:17  <isaacs>and that approach is insufficient for zlib
22:24:32  <dominictarr>isaacs, see http://npm.im/through
22:24:38  <isaacs>another paradigm filter stream is crypto
22:24:41  <Raynos>isaacs: do you want me to implement zlib as a through stream ?
22:24:43  <dominictarr>that is my baseclass for filter streams.
22:24:53  <Raynos>a filter stream does not have to be a synchronous filter
22:25:26  <isaacs>dominictarr: so, the issue with zlib is that the output and input don't even have a 1:1 correllation
22:25:33  <dominictarr>isaacs, it would be great if crypto was a stream.
22:25:40  <isaacs>when compressing, you may write 10 times, and then get 1 chunk emitted
22:25:48  <dominictarr>isaacs, whats the problem with that?
22:26:13  <isaacs>dominictarr: in the other direction, you write one chunk, and the next write returns false, as it emits chunk after chunk of results until the buffer drains
22:26:26  <isaacs>so, in practice:
22:26:33  <isaacs>1. the write() end needs to respect the false.
22:26:47  <isaacs>2. the read() end needs to be prepared to get chunks in arbitrary sizes, at an arbitrary speed.
22:26:58  <dominictarr>agree.
22:27:02  <isaacs>3. it cannot be assumed that a "end" event will follow immediately affter .end()
22:27:10  <dominictarr>agree.
22:27:35  <dominictarr>end may be buffered because the stream is paused.
22:27:40  <Raynos>isaacs: https://github.com/Raynos/through-stream#example
22:27:45  <isaacs>you even have half-duplex cases.
22:27:53  <Raynos>what's half duplex
22:28:10  <isaacs>half-duplex = we used to be duplex, but now one direction is closed
22:28:12  <Raynos>a through-stream for me is a simple stream which has a shared buffer between the write & read methods
22:28:22  <dominictarr>Raynos, it's when one side of the stream ends, but the other side can keep on writing
22:28:37  <isaacs>dominictarr: well, with filters, that's actually the part you *don't* typically have.
22:28:44  <isaacs>dominictarr: but you do have the revers
22:28:50  <isaacs>(no longer writable, but can keep reading)
22:29:24  <Raynos>isaacs: a through stream is a stream which when written to makes some or no manipulation to the underlying Buffer and when read from simply returns n bytes from the Buffer
22:29:36  <dominictarr>isaacs, that couldn't have any effect on any thing.
22:29:38  <isaacs>Raynos: zlib can't be safely done in place
22:29:51  <isaacs>you need separate input and output buffers.
22:29:59  <Raynos>isaacs: why?
22:30:07  <isaacs>Raynos: consider a file that is 100GB of "A"
22:30:12  <Raynos>oh yeah I see your point
22:30:17  <isaacs>Raynos: now you compress that file. and it's now tiny
22:30:19  <Raynos>you need a seperate window buffer
22:30:24  <isaacs>now you take the compressed bit, and decompress it
22:30:25  <Raynos>or a list of windows
22:30:47  <isaacs>you write the puny 256 buffer or whatever, and zlib goes, "Ok, this says to emit 'A' for the next 100GB bytes. Get ready..."
22:31:21  <isaacs>similar issues exist for crypto
22:31:26  <dominictarr>isaacs, half-duplex streams are broke at the moment
22:31:37  <isaacs>dominictarr: all streams are broke at the moment :)
22:31:43  <isaacs>we just have a lot of duct tape everywhere :)
22:31:55  <isaacs>wrapped around all our pipes
22:31:58  <isaacs>or "ducts" if you will
22:32:06  <dominictarr>yeah, one bit is this https://github.com/joyent/node/blob/master/lib/stream.js#L109
22:32:26  <isaacs>yeah
22:32:40  <isaacs>luckily no one actually uses half-duplexes that muhc
22:33:00  <isaacs>but we had to work around a bunch of bullshit in the zlib binding to not fall apart on that
22:33:02  <dominictarr>I was just implementing something that needed that this morning actually!
22:33:33  <dominictarr>but I decided to assume a telephone - where either side hangs up, but you say "goodbye" first.
22:33:52  <isaacs>yep
22:33:54  <isaacs>it's a pita
22:33:54  <dominictarr>isaacs, anyway, through streams are easy.
22:33:58  <isaacs>right
22:34:02  <dominictarr>what is the problem with duplex streams.
22:34:02  <isaacs>but not all filters are through streams.
22:34:13  <isaacs>so, the issue is that we have all these fucking fields and magic bs things.
22:34:31  <isaacs>water marks and "_flowing=true" and "_dests=[one,two,three]"
22:34:42  <isaacs>and you have to be careful not to mess with any of htem.
22:34:45  <isaacs>so, i'm thinking...
22:34:50  <dominictarr>I'm surprised you put in multipiping.
22:34:50  <isaacs>why not have a _readableOptions
22:35:07  <isaacs>at least then it's only one thing to mess with
22:35:09  <dominictarr>that could have just been a tee stream.
22:35:40  <isaacs>dominictarr: yeah, but a bunch of folks pointed out real-world cases that would break without multipipe
22:35:46  <isaacs>and i has a compassion for users.
22:35:59  <dominictarr>isaacs, link?
22:36:16  <dominictarr>I mean bash uses tee, whats the problem?
22:37:30  <isaacs>dominictarr: i'd rather not push a change that's going to be a pita for voxer or joyent.
22:37:44  <dominictarr>right.
22:38:09  <isaacs>dominictarr: mostly it's logging stuff
22:38:22  <isaacs>but i think we can get most use cases covered even better with a progress event of some sort
22:38:34  <isaacs>or maybe, if you have something piping, we can emit "data" events
22:38:36  <dominictarr>maybe a have a monkey patch that cleverly installs itself.
22:38:51  <isaacs>since, at that point, the draw will make sure you get all the way through it eventually
22:39:12  <isaacs>so, the issue with water marks, is that i'd also like them for writable streams
22:39:29  <isaacs>and then there was this collision of key names.
22:39:35  <isaacs>and also buffer and length and so on
22:39:43  <dominictarr>so you know how much you have filled the buffer?
22:39:49  <isaacs>sure
22:40:48  <dominictarr>then you could do a lazy pipe, because you could see whether the stream is already "paused"
22:41:23  <isaacs>yeah
22:41:39  <isaacs>of course, now we have three "pipe" implementations:
22:41:43  <isaacs>1. the original old legacy one
22:41:52  <isaacs>2. the new write() while read() one
22:41:54  <Raynos>dominictarr: What's a tee stream?
22:42:01  <dominictarr>Raynos, man tee
22:42:06  <isaacs>3. the monkey-patched pipe() after already piping one
22:42:22  <isaacs>Raynos: think actual plumbing pipes
22:42:24  <Raynos>I find multipiping important
22:42:28  <isaacs>one in, multiple out
22:42:38  <dominictarr>well, the legacy pipe was already good at multipiping
22:42:40  <Raynos>otherwise I need some kind of split stream
22:42:52  <isaacs>dominictarr: yeah, but it sucked at everythign else :)
22:42:54  <Raynos>and then I would end up always using the split stream
22:43:26  <dominictarr>isaacs, a lot of people complained about pause()
22:43:45  <dominictarr>you mean that?
22:44:05  <isaacs>dominictarr: well, more to the point, the existing readable stream interface sucks at eerythign else
22:44:12  <isaacs>dominictarr: pause doesn't pause, etc.
22:44:38  <dominictarr>maybe it's just that I got good at streams, but I really didn't have a problem with the old interface.
22:44:53  <isaacs>dominictarr: i'm pretty good at streams, but for lowlevel stuff, they really kind of suck
22:44:59  <isaacs>lots of edges.
22:45:33  <Raynos>dominictarr: I started writing the new readable streams, they are easy to write.
22:45:49  <dominictarr>yeah, that is why I spent so much time looking for common abstractions to make implementing streams easier.
22:45:52  <Raynos>I havn't had the need to create instances of PauseStream and BufferStream everywhere anymore
22:46:11  <dominictarr>like, take an async function, make it into a stream.
22:46:15  <Raynos>nor have I had the need to write duplex streams recently
22:49:20  <dominictarr>isaacs, streams have too many states. that makes them hard. maybe if we can reduce the number of states...?
22:50:17  <Raynos>dominictarr: I've reduced complexity by having only readable & writable streams and then wrapping the readable or writable stream with the filter.
22:50:31  <Raynos>Then there is only one input, one output and one pipe. Anything else is a transformation
22:50:40  <dominictarr>Raynos, yeah. that isn't good enough for core though.
22:50:45  <dominictarr>that adds overhead.
22:51:18  <Raynos>I doubt it does, you can benchmark it against pipes
22:51:42  <Raynos>net.createServer(function (sock) { new Gzip(sock).pipe(sock) })
22:52:01  <Raynos>monkey patching read is cheaper then having two pipies
22:52:04  <Raynos>pipes*
22:52:22  <dominictarr>Raynos, I've benched stuff, like passing through extra naked through streams makes a difference.
22:52:50  <dominictarr>that you can notice, if you are doing heavy IO.
22:53:18  <isaacs>dominictarr: so, "given a read(n, cb)" make a readable stream
22:53:26  <isaacs>dominictarr: that's effectively what Readable is
22:53:37  <dominictarr>that should do it for fs.
22:53:44  <isaacs>dominictarr: and for tcp
22:53:54  <dominictarr>I implemented https://github.com/dominictarr/fs-reverse/blob/master/index.js like that.
22:54:03  <isaacs>dominictarr: remember, on windows, there's no sync read() on tcp streams
22:54:15  <isaacs>and, you need to space out the actual syscalls
22:54:28  <isaacs>so _read(n, cb) is only ever called with 16*1024 or something
22:54:38  <isaacs>and then read(n) is how you actually consume that buffer.
22:55:08  <isaacs>and when it gets below some level, you do another _read(n, cb)
22:56:00  <dominictarr>that makes sense.
22:56:35  <dominictarr>no way to know where the level should be though.
22:57:20  <dominictarr>the hard thing about streams is testing them. I'd want to produce elegant reference implementations, and then ugly performant implementations.
22:57:44  <substack>awesome that taiwan happens to have the same kind of electrical outlet as north america
22:57:57  <substack>seeing what shanghai uses
22:58:37  <dominictarr>substack, in cambodia/thailand they where actually multiadapters and nz/aus plugs ALSO WORK
22:58:46  <dominictarr>I think euro works too.
22:59:21  <dominictarr>just not UK/ireland... the world's ugliest adapters.
22:59:52  <substack>oh goodness it looks like china uses all the possible adapters
22:59:54  <substack>at once
23:00:09  <dominictarr>china ++
23:00:21  <substack>http://www.travelchinaguide.com/essential/electricity.htm
23:00:25  <dominictarr>well, they manufacture them all.
23:00:37  <substack>true
23:01:11  <substack>this is crazy: http://www.madaboutshanghai.com/2005/11/power_outlets_i.html
23:02:16  * tphummeljoined
23:02:31  <substack>(crazy excellent)
23:02:32  <dominictarr>isaacs, watermarks add an extra layer between read() and and actual syscall.
23:02:47  <dominictarr>are you talking about anticipating the next call?
23:02:59  <isaacs>dominictarr: yes, watermarks add an extra layer betwee read() and an actual syscall
23:03:02  <isaacs>dominictarr: that's by design
23:03:05  <substack>china is like a gigantic kludge of a distributed system
23:03:09  <substack>in the best possible way
23:03:14  <isaacs>dominictarr: you don't want to be calling read(2) for 4 bytes.
23:03:19  <isaacs>that's very expensive.
23:03:53  <dominictarr>substack, thailand/cambodia has just the top half of that socket in the photo - but I still managed to jam a NZ plug in.
23:04:18  <isaacs>dominictarr: so yes, you anticipate the syscall
23:04:44  <isaacs>when you consume down to only 1024 bytes in the buffer, it fetches another 16kb
23:04:57  <dominictarr>just to be the devil's advocate, if you only made the syscall when the buffer was empty, how much simpler would that make the implementation?
23:05:29  <substack>china knows all about concurrent versioning
23:05:51  <substack>they've probably been using a package manager like npm for thousands of years
23:06:26  <dominictarr>substack, have you read "art of war" and / or "tao de ching"
23:06:29  <dominictarr>?
23:06:42  <isaacs>dominictarr: it'd be slower
23:06:46  <isaacs>dominictarr: and not much simpler.
23:06:51  <substack>nope
23:07:08  <isaacs>dominictarr: so, you have something like: veryFastReader.pipe(verySlowWriter)
23:07:25  <isaacs>dominictarr: or the reverse: verySlowReader.pipe(veryFastWriter)
23:07:30  <dominictarr>isaacs, like fs -> network
23:07:34  <isaacs>yeah
23:07:46  <isaacs>or, like local network -> cell phone network
23:07:50  <isaacs>much bigger diff
23:07:51  <dominictarr>slow -> fast means lots of buffering is good.
23:08:00  <dominictarr>yup
23:08:29  <substack>blarg everybody speaks english here
23:08:32  <isaacs>the worst is when you have shit where it says, "That write couldn't flush" so you pause the reader, then it does flush right away, and emits drain
23:08:44  <substack>silly rich country
23:08:51  <dominictarr>I'm guessing that fast -> slow would not make so much difference.
23:09:01  <isaacs>substack: traveling the world for conferences has taught me that other countries are populated mostly by english speaking computer nerds
23:09:17  <dominictarr>substack, still, if you know hello and thank you that wins you points.
23:09:30  <substack>I have 2 pages filled with phrases
23:09:40  <substack>haven't had a chance to use them yet!
23:09:52  <dominictarr>TODO: GOTO PLACE THAT DOES NOT SPEAK ENGLISH. MAYBE FRANCE.
23:09:53  <LOUDBOT>I THINK MY PIE IS GOING TO BE WATERY
23:10:22  <substack>france actually can speak english, they just militantly choose not to
23:10:38  <dominictarr>I respect them for that.
23:10:44  <substack>same
23:11:33  <dominictarr>you don't even need to speak at all for most important things, eating, finding a place to sleep, etc.
23:11:57  <dominictarr>99% of what people say is completely unnecessary.
23:11:59  <Raynos>dominictarr: I'm giving a talk at LNUG. Thinking about doing "reactive programming with streams". Any better suggestions?
23:12:25  <dominictarr>Raynos, that is good. I'm gonna do "writing custom streams"
23:12:50  * substackdoing a survey of streaming libs at lisbon
23:13:18  <st_luke>crap I have to buy a plane ticket
23:13:23  <st_luke>that shit expensive
23:13:28  <dominictarr>hitch hike.
23:13:38  <dominictarr>substack, what about dublin?
23:13:41  <st_luke>on a boat
23:13:50  <dominictarr>st_luke, where do you live?
23:13:55  <st_luke>dominictarr: nyc
23:14:02  <substack>dominictarr: something more similar to what I'm doing for shanghai about how littleness is good
23:14:14  <dominictarr>it's usually cheaper if you brought the ticket yesterday.
23:14:15  <substack>except with leprechauns instead of dragons
23:14:26  <dominictarr>haha.
23:14:37  <dominictarr>that is very tao de ching
23:15:25  <dominictarr>{ruling a country, writing a module} is like flying a small fish.
23:15:39  <dominictarr>(don't poke it too much)
23:16:08  <substack>also I found this: http://zh.wikipedia.org/zh-cn/Unix%E5%93%B2%E5%AD%A6
23:17:50  * substacklistening to 90s alt country
23:18:05  <dominictarr>heh, it links to "elements of style" a book I brought, but never got around to reading.
23:18:06  <substack>most unfitting music possible
23:18:13  <substack>for this venue
23:18:24  <dominictarr>it's not about unix, I'm pretty sure.
23:18:37  <substack>yeah
23:18:51  <substack>it's just that the people building unix stumbled upon this design philosophy that works
23:22:56  <dominictarr>people who design things always have very wide interests.
23:23:18  <substack>I said this on twitter but it's so true
23:23:18  <dominictarr>they turn over every rock, searching for answers.
23:23:24  <substack>bell labs was the bauhaus of computing
23:23:45  <substack>a bunch of smart people bouncing ideas off each other for a brief time and getting to the essence of design
23:23:58  <substack>yes, thoroughly exploring the solution space
23:24:07  <substack>not getting stuck on local maxima
23:24:22  <st_luke>substack: sometimes I wish I could live through that era
23:24:35  <st_luke>if I had a holodeck I would spend a lot of time at bell labs
23:24:36  <substack>st_luke: well then the nazis came and they all fled to america
23:24:44  <substack>oh hah bell labs
23:25:41  <st_luke>is mbalho active?
23:25:42  <substack>st_luke: just watch that at&t archive footage on a loop
23:26:14  <substack>it's amazing that a company as thoroughly evil as at&t could create something as beautiful and benevolent as unix
23:26:25  <isaacs>dominictarr: large swaths of south america have very few english speakers.
23:26:36  <substack>maybe extreme evil seeks extreme goodness or some such platitude
23:27:00  <isaacs>substack: i think they actually got eviler when the govt forced them to split up
23:27:09  <st_luke>substack: sometimes in these giant companies with too much money people gather and realize they can use the resources to make something special
23:27:12  <st_luke>never lasts long though
23:27:13  <substack>the evil got concentrated, perhaps
23:27:25  <substack>like terrorist splinter cells
23:27:26  <dominictarr>substack, st_luke but isn't oakland that right now?
23:27:46  <isaacs>ok, i'm gonna pull the high/low watermark from the Readable stream
23:27:54  <st_luke>dominictarr: from my experience in oakland (which is fairly limited), I would say it feels like that sort of place
23:27:58  <isaacs>fs and net streams will just bolt that bit on
23:28:38  <dominictarr>isaacs, ++
23:29:03  <st_luke>if I were to move somewhere in the US after I'm done in new york it would probably be oakland
23:29:47  <dominictarr>"after I'm done in new york"? I've never been to nyc, but listening to frank sinatra, how is that possible?
23:30:18  <substack>the northeast is my only remaining US blackspot
23:30:31  <isaacs>st_luke: yes.
23:30:41  <isaacs>st_luke: oakland is the correct place to end up
23:31:08  <isaacs>st_luke: after new york, sf might be an easier transition, though
23:31:12  <st_luke>substack: the northeast is a nice place for the geography
23:31:14  <isaacs>unless you find yourself fed up with cities in general.
23:31:19  <dominictarr>there is also I think an important factor; freedom. I've always done my best work when the boss was on holiday.
23:31:25  <substack>been to 43/50 of the states
23:31:34  <isaacs>but like, if you miss the smell of pee and the oppressive weight of concrete, oakland might be a shock to the system.
23:31:50  <dominictarr>I heard that when they where writing unix, they where _meant_ to be writing a word processor.
23:31:52  <st_luke>isaacs: that pee smell never gets old
23:32:24  <isaacs>st_luke: i've noticed that the thing that the mission and the bronx have in common is that people who *live* there claim "it's not that bad"
23:32:24  <substack>dominictarr: that might explain the spell checker that kernighan busts out in that 1982 video
23:32:29  <isaacs>and people who visit gag constantly.
23:33:05  <st_luke>isaacs: i was really disappointed with the mission, I had the wrong idea of what it would be
23:33:19  <isaacs>st_luke: yeah, it's probably the most overhyped place i've ever been.
23:33:24  <dominictarr>substack, yeah. and maybe the whole "streams of text" thing too.
23:33:33  <isaacs>the thing that's good about it is that there are a lot of people socializing all the time.
23:33:37  <isaacs>many bars and shit.
23:33:45  <isaacs>(not "and stuff", i mean literal feces)
23:34:07  <dominictarr>once, I googled "hipster bars" and went out in the mission, but I had way more adventures coming back to oakland on the BART
23:34:22  <st_luke>I live in a neighborhood with a whole lot of bars and hipsterish people right now
23:34:31  <st_luke>that's more or less what people in brooklyn do
23:35:35  <substack>I hardly ever go to bars
23:35:47  <substack>only when I know some people who will be there
23:36:04  <dominictarr>go to a bar, look around, is anyone having fun? no. go home.
23:36:19  <substack>seems like a good algorithm
23:36:23  <dominictarr>most people have no idea why they go to bars.
23:36:56  <substack>I find it's much easier to strike up conversations at coffee shops
23:37:02  <substack>since people tend to be working on interesting things
23:37:16  <substack>and it's quiet enough to talk
23:37:16  <dominictarr>and not so loud.
23:37:23  <st_luke>the thing that looks cool about the bay area and oakland is people doing tech meetups for the sake of just getting computers out and playing around on them
23:37:31  <st_luke>maybe building something, in a group or individually
23:37:35  <substack>like once at awaken I saw a guy working on some category theory stuff
23:37:54  <st_luke>tech meetups in nyc are about the networking aspect more than anything, not people sitting around discovering things that their computer can do
23:38:01  <substack>and my passing familiarity with it via haskell was enough to have some back and forth
23:38:29  <st_luke>substack: that's a cool experience
23:38:40  <substack>berkeley is really great for that stuff
23:38:44  <substack>with all the students around
23:39:00  <st_luke>reminds me of the story I read about the guy who ran into someone at a coffee shop who invented the computer
23:39:12  <substack>arguing about economics or sociology experiments
23:39:19  <st_luke>yeah
23:39:38  <st_luke>it's why I like to go to hack events and not participate in the competition part
23:39:44  <substack>oakland has more of the artist/radical vibe which is neat too
23:40:08  <substack>I see folks like boots riley just walking around down by the plaza
23:40:34  <substack>and then protests shut down downtown once every few months
23:42:16  <st_luke>yeah, I think I might end up there sometime
23:42:24  <st_luke>but I would like to try a few months in Europe first
23:42:30  <substack>rad
23:42:32  <dominictarr>oh, man. he has an album named "genocide & juice"
23:42:34  <substack>crazy adventures
23:43:38  <st_luke>yeah, maybe Berlin and Russia if I can save up some money
23:45:04  <isaacs>substack: the loudness of bars is intentional.
23:45:26  <substack>isaacs: so boring people don't get discovered?
23:45:37  <isaacs>substack: the purpose is to make you have to lean in, and to increase the difficulty of speaking to the point where we detach our analytical minds and start speaking mostly lower-level primate
23:45:55  <substack>I hate everything about that. >_<
23:46:00  <isaacs>noise -> physical closeness, lack of linguistic precision
23:46:07  <isaacs>couple with alcohol, and the effect increases.
23:46:15  <dominictarr>just stop talking and use body language.
23:46:19  <isaacs>exactly
23:46:21  <isaacs>or, learn ASL
23:46:33  <dominictarr>charades is more fun.
23:46:42  <isaacs>going to bars with marisa is great, because she's *always* hard of hearing, read lips really well, and we can sign to each other.
23:46:45  <st_luke>I like when bars get closed down and are used for small conferences
23:46:47  <isaacs>it's like having a secret language.
23:47:17  <dominictarr>srsly, charades is great fun. but be warned, you start to invent your own language.
23:47:28  <substack>st_luke: especially when they turn the music way down
23:48:07  <st_luke>yeah
23:48:54  <st_luke>actually I think outside of conferences I hardly ever drink
23:50:02  <st_luke>hopefully synthehol will be invented soon
23:50:10  <st_luke>or marketed
23:52:42  <dominictarr>I heard there is a pill that near-instantly makes you sober
23:53:22  <dominictarr>but it was too much of a legal minefield, you'd still be legally drunk due to blood alcohol.
23:53:57  <substack>dominictarr: oh there was a race condition that somebody fixed recently in browserify
23:54:03  <substack>could be related to that bug you just closed
23:54:05  <dominictarr>I saw that.
23:54:18  * substackis terrible about closing issues
23:54:42  <dominictarr>you are spread too thin, too many modules.
23:54:49  <dominictarr>you need to delegate.
23:55:41  <substack>it's only a handful of modules that give a ton of noise
23:56:00  <substack>browserify is super noisy
23:57:15  <isaacs>substack: i hear ya
23:57:31  <isaacs>substack: every so often i take a saturday to close issues in npm
23:57:37  <isaacs>usually there's 10 or 20 dupes of every actual issue
23:57:46  <isaacs>or things that people figured out, but never came back to resolve
23:58:01  <st_luke>those are the worst
23:58:02  <isaacs>nominating helpers is winful
23:58:20  <substack>I have >100 modules that pretty much have no noise at all
23:58:31  <substack>those tend to be the very tiny hyper-focused modules with lots of tests
23:58:50  <st_luke>sometimes I don't like opening issues for what I think might be silly things, but other times I like opening issues on modules released by companies that look like they do it partially for marketing reasons
23:58:53  <substack>best kind of module
23:59:10  <substack>people who complain about modules being too small do not have this problem for the most part