| 00:00:01
| * ircretary | quit (Remote host closed the connection) |
| 00:00:12
| * ircretary | joined |
| 00:00:30
| * Maciek416 | joined |
| 00:05:30
| * Maciek416 | quit (Remote host closed the connection) |
| 00:08:31
| * Maciek416 | joined |
| 00:09:23
| * Maciek416 | quit (Remote host closed the connection) |
| 00:39:04
| <DTrejo> | ircretary is handy |
| 00:39:04
| <ircretary> | DTrejo: I'm not sure what to do with that command. Ask for help in PM. |
| 00:43:59
| * i_m_ca | joined |
| 00:44:37
| * DTrejo | quit (Remote host closed the connection) |
| 00:47:00
| * DTrejo | joined |
| 01:06:36
| * timoxley | joined |
| 01:13:18
| * ins0mnia | quit (Ping timeout: 264 seconds) |
| 01:14:37
| * calvinfo | quit (Quit: Leaving.) |
| 01:24:09
| * calvinfo | joined |
| 01:28:38
| * ralphtheninja | quit (Ping timeout: 240 seconds) |
| 01:41:07
| * mikolalysenko | joined |
| 01:50:15
| * Maciek416 | joined |
| 01:51:39
| * defunctzombie | changed nick to defunctzombie_zz |
| 02:09:15
| * thlorenz | joined |
| 02:10:49
| * DTrejo | quit (Remote host closed the connection) |
| 02:14:26
| * timoxley | quit (Remote host closed the connection) |
| 02:19:08
| * AvianFlu | quit (Remote host closed the connection) |
| 02:19:36
| * AvianFlu | joined |
| 02:26:09
| * jxson | quit (Remote host closed the connection) |
| 02:26:17
| * DTrejo | joined |
| 02:28:07
| * mikolalysenko | quit (Ping timeout: 260 seconds) |
| 02:36:01
| * michaelrhodes | joined |
| 02:38:02
| * DTrejo | quit (Remote host closed the connection) |
| 02:48:29
| * thlorenz | quit (Remote host closed the connection) |
| 02:48:50
| * AvianFlu | quit (Remote host closed the connection) |
| 02:49:18
| * AvianFlu | joined |
| 02:58:13
| * ralphtheninja | joined |
| 02:59:47
| * defunctzombie_zz | changed nick to defunctzombie |
| 03:03:34
| * AvianFlu | quit (Remote host closed the connection) |
| 03:04:49
| * AvianFlu | joined |
| 03:08:30
| * FireFly | quit (Excess Flood) |
| 03:08:48
| * FireFly | joined |
| 03:10:12
| * fallsemo | joined |
| 03:11:16
| * tilgovi | quit (Ping timeout: 256 seconds) |
| 03:26:29
| * jxson | joined |
| 03:31:18
| * jxson | quit (Ping timeout: 264 seconds) |
| 03:31:50
| * i_m_ca | quit (Ping timeout: 240 seconds) |
| 03:48:31
| * fallsemo | quit (Quit: Leaving.) |
| 03:50:41
| * i_m_ca | joined |
| 04:04:53
| * timoxley | joined |
| 04:06:58
| * feross | joined |
| 04:23:49
| * dguttman | quit (Quit: dguttman) |
| 04:43:06
| * i_m_ca | quit (Ping timeout: 264 seconds) |
| 04:56:50
| * feross | quit (Quit: feross) |
| 05:27:32
| * timoxley | quit (Remote host closed the connection) |
| 05:47:27
| * Maciek416 | quit (Remote host closed the connection) |
| 05:51:15
| * shama | quit (Remote host closed the connection) |
| 06:24:58
| * Maciek416 | joined |
| 06:33:03
| * Maciek416 | quit (Remote host closed the connection) |
| 06:52:54
| * shama | joined |
| 06:59:38
| * shama | quit (Ping timeout: 264 seconds) |
| 07:22:46
| * defunctzombie | changed nick to defunctzombie_zz |
| 07:26:46
| * michaelrhodes | quit (Quit: michaelrhodes) |
| 07:36:37
| * owenb_ | quit (Ping timeout: 248 seconds) |
| 07:36:43
| * defunctzombie_zz | changed nick to defunctzombie |
| 07:43:37
| * dominictarr | joined |
| 08:14:43
| * timoxley | joined |
| 08:22:42
| * jcrugzz | quit (Ping timeout: 264 seconds) |
| 08:23:02
| * ehd | quit (Ping timeout: 240 seconds) |
| 08:27:19
| * ehd | joined |
| 08:33:17
| * peutetre | joined |
| 08:46:26
| * jlord | quit (Ping timeout: 264 seconds) |
| 08:47:04
| * jlord | joined |
| 09:02:54
| * peutetre | quit (Ping timeout: 264 seconds) |
| 09:36:04
| * owenb_ | joined |
| 09:38:50
| * dominictarr | quit (Quit: dominictarr) |
| 09:39:04
| * defunctzombie | changed nick to defunctzombie_zz |
| 09:49:27
| * defunctzombie_zz | changed nick to defunctzombie |
| 09:50:14
| * dominictarr | joined |
| 10:03:07
| * defunctzombie | changed nick to defunctzombie_zz |
| 10:09:42
| * ralphtheninja | quit (Ping timeout: 264 seconds) |
| 10:16:11
| * fuz | joined |
| 10:26:43
| * ins0mnia | joined |
| 10:50:07
| * dominictarr | quit (Quit: dominictarr) |
| 10:52:35
| * fuz | quit (Ping timeout: 260 seconds) |
| 11:00:44
| * fuz | joined |
| 11:17:26
| * dominictarr | joined |
| 11:59:27
| <substack> | https://github.com/substack/node-hat#how-to-make-a-hat |
| 12:30:14
| * fallsemo | joined |
| 12:33:26
| * ins0mnia | quit (Ping timeout: 240 seconds) |
| 12:33:31
| * yorick | joined |
| 12:52:12
| * mikolalysenko | joined |
| 13:00:40
| * ralphtheninja | joined |
| 13:06:54
| * vitorpacheco | joined |
| 13:08:54
| <mbalho> | substack: lol |
| 13:09:00
| <mbalho> | substack: i think this is a really nice intro http://blakeembrey.com/articles/introduction-to-browserify/ |
| 13:10:30
| <substack> | mbalho: yep, I saw that |
| 13:10:48
| * fuz | quit (Ping timeout: 256 seconds) |
| 13:17:51
| * mikolalysenko | quit (Ping timeout: 259 seconds) |
| 13:31:50
| * mikolalysenko | joined |
| 13:40:19
| * vitorpacheco | quit (Quit: Saindo) |
| 13:47:55
| * fallsemo | quit (Ping timeout: 246 seconds) |
| 13:57:11
| * fallsemo | joined |
| 13:57:35
| * i_m_ca | joined |
| 13:57:38
| * shama | joined |
| 14:01:52
| * fallsemo | quit (Ping timeout: 256 seconds) |
| 14:01:55
| * shama | quit (Ping timeout: 246 seconds) |
| 14:21:32
| * shama | joined |
| 14:30:30
| * i_m_ca | quit (Ping timeout: 264 seconds) |
| 14:39:34
| * Maciek416 | joined |
| 15:03:48
| * shama | quit (Remote host closed the connection) |
| 15:04:43
| * ins0mnia | joined |
| 15:16:58
| * rpflo | changed nick to ryanflorence |
| 15:28:17
| * kriskowal | joined |
| 15:34:48
| * shama | joined |
| 15:37:54
| * fallsemo | joined |
| 15:43:02
| * shama | quit (Ping timeout: 240 seconds) |
| 15:44:26
| * mikolalysenko | quit (Ping timeout: 256 seconds) |
| 15:44:38
| * fallsemo | quit (Ping timeout: 264 seconds) |
| 16:02:17
| * dominictarr | quit (Quit: dominictarr) |
| 16:06:41
| * Maciek416 | quit (Remote host closed the connection) |
| 16:08:17
| * shama | joined |
| 16:10:16
| * yorick | quit (Remote host closed the connection) |
| 16:12:10
| * feross | joined |
| 16:12:28
| * i_m_ca | joined |
| 16:15:07
| * dguttman | joined |
| 16:21:19
| * timoxley | quit (Remote host closed the connection) |
| 16:21:41
| * timoxley | joined |
| 16:25:59
| * fallsemo | joined |
| 16:30:19
| * fallsemo | quit (Ping timeout: 246 seconds) |
| 16:51:38
| * mikolalysenko | joined |
| 16:56:38
| * mikolalysenko | quit (Ping timeout: 264 seconds) |
| 16:58:58
| * fuz | joined |
| 16:59:15
| * jcrugzz | joined |
| 17:02:42
| * feross | quit (Quit: feross) |
| 17:17:51
| * thlorenz | joined |
| 17:24:57
| * dominictarr | joined |
| 17:26:07
| * tmcw | joined |
| 17:27:29
| * feross | joined |
| 17:36:36
| * tmcw | quit (Remote host closed the connection) |
| 17:37:03
| * tmcw | joined |
| 17:42:06
| * tmcw | quit (Ping timeout: 264 seconds) |
| 17:48:20
| * i_m_ca | quit (Quit: Lost terminal) |
| 18:20:01
| * AvianFlu | quit (Remote host closed the connection) |
| 18:20:30
| * AvianFlu | joined |
| 18:30:10
| * thlorenz | quit (Remote host closed the connection) |
| 18:32:13
| * thlorenz | joined |
| 18:33:43
| * thlorenz | quit (Remote host closed the connection) |
| 18:35:52
| * thlorenz | joined |
| 18:38:27
| * thlorenz | quit (Remote host closed the connection) |
| 18:47:38
| * tmcw | joined |
| 18:52:27
| * tmcw | quit (Ping timeout: 260 seconds) |
| 18:54:51
| * kriskowal | quit (Quit: kriskowal) |
| 19:01:20
| * missinglink | joined |
| 19:06:56
| * fuz__ | joined |
| 19:07:15
| * fuz | quit (Ping timeout: 260 seconds) |
| 19:14:12
| * defunctzombie_zz | changed nick to defunctzombie |
| 19:22:34
| <dominictarr> | creationix: hey, what if simple-streams also had a isEnd() |
| 19:22:45
| <dominictarr> | then you could have undefined values |
| 19:23:03
| <dominictarr> | and undefined wouldn't be a sentinel |
| 19:23:16
| <dominictarr> | I could stop using cb(true) as end |
| 19:23:23
| <dominictarr> | and could make pull streams compatible |
| 19:25:34
| <creationix> | dominictarr, why do you need undefined as values? |
| 19:25:50
| <dominictarr> | because undefined happens |
| 19:25:52
| <dominictarr> | quite often |
| 19:26:03
| <dominictarr> | all the array ops etc |
| 19:26:10
| <dominictarr> | having it be end is surprising |
| 19:26:19
| <creationix> | hasn't bit me once yet |
| 19:26:22
| <creationix> | hmm |
| 19:26:30
| <dominictarr> | and your msgpack-js even encodes undefined |
| 19:26:57
| <dominictarr> | it's a value |
| 19:27:10
| <dominictarr> | but end is a state |
| 19:27:47
| <creationix> | if you need undefined as a value in simple-streams, simply wrap all values in an array or something |
| 19:28:12
| <dominictarr> | that is a work around |
| 19:28:27
| <creationix> | so it having to call isEnd with every value |
| 19:28:35
| <creationix> | or at least with every undefined value |
| 19:28:36
| <dominictarr> | what if, if there is no isEnd, then undefined isEnd |
| 19:29:03
| <creationix> | well you can do that today |
| 19:29:09
| <creationix> | it would be a superset of simple-streams |
| 19:29:29
| <creationix> | you would just need to update any generic stream consuming code |
| 19:29:49
| <dominictarr> | I want to make our stuff compatible |
| 19:30:07
| <creationix> | I've already got abort in the spec and I never use it in practice, I'm not sure I want to add more to the spec that I've never had a need for |
| 19:30:57
| <creationix> | so since your interface is more powerful, it can wrap mine |
| 19:31:01
| <creationix> | you just can't go the other way |
| 19:31:10
| <dominictarr> | I don't want two interfaces though |
| 19:31:31
| <creationix> | I see, and you care about those edge cases |
| 19:31:32
| <creationix> | hmm |
| 19:31:33
| <dominictarr> | with one interface no body has to choose which one to favor |
| 19:32:53
| <creationix> | so the only problem with push style was that back-pressure was tricky right? |
| 19:32:58
| <creationix> | otherwise it's a much better interface |
| 19:33:11
| <creationix> | having to know when to call pause and resume is a real pain |
| 19:33:51
| <creationix> | I guess you could put a callback on the onData event |
| 19:34:37
| <creationix> | (I'm exploring because I'm not happy with simple-stream or min-stream) |
| 19:35:17
| <creationix> | if I found an interface that I liked, I'd be willing to port js-git again. But I'm about to publish public APIs, so I need to decide soon |
| 19:35:21
| * timoxley | quit (Remote host closed the connection) |
| 19:35:37
| <dominictarr> | yeah, pull makes back pressure implicit |
| 19:35:49
| <creationix> | but push makes all sorts of other stuff really nice |
| 19:36:14
| <dominictarr> | okay, well, for me, I want just one interface |
| 19:36:35
| <dominictarr> | that is reasonably good at everything |
| 19:36:42
| <dominictarr> | and backpressure is the hard part |
| 19:36:47
| <dominictarr> | so, optimise for that. |
| 19:36:53
| <creationix> | did you ever take the time to understand https://github.com/Gozala/reducers |
| 19:36:53
| <creationix> | he tried to explain it to me, but I didn't get it |
| 19:37:15
| <dominictarr> | so, I think I understand it |
| 19:37:25
| <dominictarr> | actually, I was an influence |
| 19:37:27
| <creationix> | the other problem I have with callback-based pull streams is the lack of tail recursion blows the stack if you're not careful |
| 19:37:37
| <dominictarr> | Oh, I have a thing for that |
| 19:37:42
| <dominictarr> | that checks if the cb is sync |
| 19:37:43
| <creationix> | and it really destroys call stacks |
| 19:37:47
| <dominictarr> | and iterates instead |
| 19:37:47
| <creationix> | everything is inverted |
| 19:37:56
| <creationix> | yeah, I've done trampolines before |
| 19:38:05
| <dominictarr> | https://npmjs.org/package/looper |
| 19:38:06
| <creationix> | my generic simple-stream tools have them |
| 19:38:55
| <dominictarr> | reducers is a nice idea, but It tried to be too clojure |
| 19:39:12
| <dominictarr> | It relied on types (as end, error sentinels) |
| 19:39:29
| <dominictarr> | which had problems with modules |
| 19:39:29
| <creationix> | could we take the general idea and simplify it though |
| 19:39:38
| <creationix> | instanceof is terrible in node |
| 19:39:41
| <creationix> | npm duplicates too much |
| 19:39:42
| <dominictarr> | exactly |
| 19:40:08
| <dominictarr> | don't use singletons in node unless you are prepared to make them global |
| 19:40:22
| <dominictarr> | or pass them around explicitly |
| 19:40:32
| <creationix> | yeah, in js-git I pass them around |
| 19:41:50
| <dominictarr> | creationix: I experimented with this https://npmjs.org/package/strm |
| 19:41:55
| <dominictarr> | before pull-streams |
| 19:42:41
| <dominictarr> | but pull is way better |
| 19:44:28
| <creationix> | so I think the idea gozala told me was to use push streams, but only pause-resume the source directly |
| 19:44:45
| <creationix> | instead of pausing your immediate parent and hoping it propagates properly |
| 19:46:59
| <dominictarr> | I have a slightly different interpretation of how reducers do that |
| 19:47:08
| <dominictarr> | but I agree that node streams do that badly |
| 19:47:21
| <dominictarr> | and both pull and reducers do it better |
| 19:47:42
| <creationix> | push style is definitely better for writing parsers |
| 19:47:48
| <dominictarr> | in both, you can have a simple transform that propagates pause state instantly |
| 19:48:09
| <creationix> | function parser(emit) { return function (item) { ... }} |
| 19:48:10
| <creationix> | I love that interface |
| 19:48:11
| <creationix> | but no back-pressure at all |
| 19:48:15
| <dominictarr> | it's easy to wrap a pusher into a puller |
| 19:48:28
| <creationix> | yep https://github.com/creationix/push-to-pull |
| 19:48:46
| <dominictarr> | push is also good for UI stuff |
| 19:48:56
| <dominictarr> | where there is no concept of back pressure anyway |
| 19:49:07
| <dominictarr> | user input is not very fast... |
| 19:49:15
| <dominictarr> | …and can't be slowed down |
| 19:49:30
| <creationix> | right |
| 19:49:46
| <dominictarr> | but, that is the easy case |
| 19:49:48
| <creationix> | and I love the interface for protocol parsers because they can ignore error handling |
| 19:49:53
| <creationix> | just throw |
| 19:50:09
| * thlorenz | joined |
| 19:50:18
| <dominictarr> | better to optimize for the difficult case, because you'll spend more time there, in the end. |
| 19:50:21
| <creationix> | (unless your parser does I/O and throws in a callback) |
| 19:50:39
| <creationix> | is pause/resume really that hard though? |
| 19:50:54
| <creationix> | I've spent a ton of time in js-git implementing simple streams |
| 19:51:15
| <creationix> | and abort is almost never actually tested and pull streams have a nasty edge case of their own |
| 19:51:25
| <creationix> | what if the user calls read a second time before you call the callback |
| 19:51:41
| <dominictarr> | that isn't allowed |
| 19:51:45
| <creationix> | you can either check for that and emit an error or queue up the read callbacks |
| 19:51:59
| <creationix> | so you just assume they will never do that |
| 19:52:06
| <creationix> | and if they do strange things happen? |
| 19:52:20
| <creationix> | because I have been bitten by that one |
| 19:52:20
| <dominictarr> | well, you could easily wrap a stream in something to enforce that contract. |
| 19:52:41
| <creationix> | yeah, but that's more layers and uglier stack traces |
| 19:53:03
| <dominictarr> | 1 more layer, that you can chuck in when things get hairy |
| 19:53:04
| <creationix> | my biggest issue with cps style is the worthless stack traces |
| 19:53:08
| * yorick | joined |
| 19:53:31
| <dominictarr> | I just don't think you'll be able to make something much easier than pull streams that can still do everything |
| 19:54:40
| * kriskowal | joined |
| 19:55:01
| * timoxley | joined |
| 19:55:44
| <creationix> | what if data was pull, but error and end were push? |
| 19:56:00
| <creationix> | you don't need back-pressure for error and end events |
| 19:58:16
| * timoxley | quit (Remote host closed the connection) |
| 19:58:45
| <creationix> | dominictarr, https://gist.github.com/creationix/d05f75643bb0abdf2f90 |
| 19:58:56
| <creationix> | that would actually be a lot easier to consume and to produce |
| 19:59:06
| <creationix> | just not sure where the idea of abort fits in |
| 19:59:50
| <dominictarr> | read takes a callback? |
| 20:00:02
| <creationix> | right, but a data-only callback |
| 20:00:03
| <dominictarr> | with no err arg? |
| 20:00:04
| <creationix> | no err param |
| 20:00:06
| <dominictarr> | ight |
| 20:01:02
| <dominictarr> | how would I pipe it? |
| 20:01:39
| <creationix> | you mean a stream transform? |
| 20:01:41
| <creationix> | just a sec... |
| 20:04:47
| <creationix> | dominictarr, https://gist.github.com/creationix/d05f75643bb0abdf2f90#file-filter-js |
| 20:06:55
| <creationix> | or if they pass in the listeners https://gist.github.com/creationix/d05f75643bb0abdf2f90#file-filter-2-js |
| 20:07:19
| <dominictarr> | okay, sure. looks reasonable |
| 20:07:30
| <dominictarr> | I just don't think there is a magic configuration that is easier |
| 20:07:36
| <dominictarr> | streams are just this hard |
| 20:07:48
| <dominictarr> | this will be hard too |
| 20:08:01
| <dominictarr> | all you can do is pick one and get on with it. |
| 20:08:26
| <creationix> | I agree that pull-streams are probably better for the data channel |
| 20:08:33
| <creationix> | people just don't get pause/resume right |
| 20:08:43
| <dominictarr> | the only way to make it easier is to do something that can't do everything |
| 20:08:44
| <creationix> | which is sad, that performs better and is nicer on stack traces |
| 20:08:55
| <creationix> | push without backpressure is easy! |
| 20:09:20
| <dominictarr> | yeah, but it's only easier until you need backpressure |
| 20:09:37
| <creationix> | right, at which point you're screwed or tack on pause/resume to the entire chain |
| 20:09:45
| <creationix> | and end up with node |
| 20:09:51
| <dominictarr> | exactly |
| 20:09:55
| <creationix> | I was there :) |
| 20:10:00
| * defunctzombie | changed nick to defunctzombie_zz |
| 20:10:27
| <dominictarr> | and then in a yeah, you'll realize that pull is simpler for back pressure and get streams 2 |
| 20:10:58
| <creationix> | and then you'll realize that push is much easier in a lot of use cases and combine both into streams 3 |
| 20:10:58
| <dominictarr> | and then realize that push was often good and so create streams3 |
| 20:11:04
| <dominictarr> | yeah |
| 20:11:05
| <creationix> | and wish event emitters should have never been in there |
| 20:11:42
| <creationix> | so you really want undefined as a value, I can do that, but not by tacking stuff onto the simple-stream inteface |
| 20:11:43
| <dominictarr> | the only solution is a new language that completely abstracts it all, and uses static analysis to decide how to connect streams |
| 20:11:51
| <creationix> | this new hybrid interface seems to work |
| 20:12:26
| <creationix> | also I have yet to find a single useful place for "abort" in js-git |
| 20:12:36
| <dominictarr> | I can show you some examples |
| 20:12:39
| <dominictarr> | where I use it |
| 20:12:40
| <creationix> | does it really need to be part of the spec or can people create side-channels manually when they need it |
| 20:12:55
| <creationix> | the spec is the core, the part that all tools gurantee to implement properly |
| 20:14:15
| <dominictarr> | here is one place I use it |
| 20:14:16
| <dominictarr> | https://github.com/dominictarr/pull-level/blob/master/index.js#L21-L25 |
| 20:14:22
| <dominictarr> | to clean up listeners when the stream ends |
| 20:14:44
| <creationix> | heh, I don't use event emitters anywhere |
| 20:14:54
| <creationix> | because I don't like cleanup |
| 20:16:12
| <dominictarr> | an event listener like thing is unavoidable in this case |
| 20:16:21
| <creationix> | that's fine |
| 20:16:27
| <creationix> | do you need the callback in abort? |
| 20:16:45
| <creationix> | I find I never care about when cleanup is done, just that it's done |
| 20:17:01
| <creationix> | and if there was an async error doing cleanup, I don't care usually |
| 20:17:27
| <creationix> | it's kind of a best-effort hint thing |
| 20:17:37
| <creationix> | the consumer is telling the source it won't be consuming anymore |
| 20:17:54
| <creationix> | it's not asking for any guarantees in return though |
| 20:18:16
| <creationix> | dominictarr, do you see "abort" the same? |
| 20:18:42
| <dominictarr> | hmm, let me think |
| 20:22:02
| <dominictarr> | I think I just gave it a cb because then a through could be really simple |
| 20:22:16
| <creationix> | ok |
| 20:22:18
| <dominictarr> | read(null, function (end, data) { cb(end, data) } |
| 20:22:19
| <dominictarr> | ) |
| 20:22:44
| <creationix> | are you ok with ES5 syntax? |
| 20:22:49
| <dominictarr> | it would still probably work if it ignored the cb |
| 20:22:50
| <creationix> | or do you need to support ES3 |
| 20:23:04
| <dominictarr> | well, I'd want it to work in current browsers |
| 20:23:20
| <creationix> | but how much browser coverage? |
| 20:23:31
| <dominictarr> | which ES5 syntax? |
| 20:23:31
| <creationix> | most have getters/setters for example |
| 20:23:49
| <creationix> | not that they would be part of the spec, but they make some idioms really nice |
| 20:24:01
| * defunctzombie_zz | changed nick to defunctzombie |
| 20:24:18
| <dominictarr> | I think avoiding setters is better |
| 20:25:32
| <creationix> | dominictarr, https://gist.github.com/creationix/d05f75643bb0abdf2f90#file-filter-js |
| 20:26:01
| <creationix> | the transform could be implemented without setters, it's just more runtime cost |
| 20:26:52
| <creationix> | like was done at https://gist.github.com/creationix/d05f75643bb0abdf2f90/dce583f548b14581a5f2ace84b8b6623717caf08#file-filter-js |
| 20:27:25
| <dominictarr> | I'm skeptical |
| 20:28:17
| <dominictarr> | why is this better than read(err, data) ? |
| 20:28:34
| <creationix> | because it seperates concernt |
| 20:28:36
| <creationix> | *concerns |
| 20:28:46
| <creationix> | different code always handles errors, data events and end events |
| 20:28:59
| <creationix> | instead of if statements all over, there are three different streams |
| 20:29:06
| * timoxley | joined |
| 20:29:31
| <dominictarr> | that is just classic streams |
| 20:29:44
| <dominictarr> | but with read(cb) |
| 20:29:51
| <creationix> | right, it's a hybrid |
| 20:29:59
| <creationix> | classic without pause/resume |
| 20:30:35
| <dominictarr> | I think the good thing about everything being past into the same cb is that you have a definite point where the state changes |
| 20:30:46
| <dominictarr> | I'll be interested to see what you come up with |
| 20:30:50
| <creationix> | also the problem with (err, data) is you can't report errors that happen between reads |
| 20:30:58
| <creationix> | suppose a socket gets disconnected, but the user isn't reading |
| 20:31:03
| <creationix> | you can't tell them till the next read |
| 20:31:07
| <dominictarr> | yes |
| 20:31:08
| <creationix> | so you have to buffer that error somewhere in a queue |
| 20:31:11
| <creationix> | it's super annoying |
| 20:31:18
| <creationix> | not that it ever happens, but it could |
| 20:31:26
| <dominictarr> | of course |
| 20:31:29
| <creationix> | and so you have to code for it, but never get to actually test that code |
| 20:31:38
| <creationix> | I hate writing code that never gets used |
| 20:32:12
| <dominictarr> | I've found it's much easier to make pull streams robust than streams1 |
| 20:32:31
| <dominictarr> | I have written much more sophisticated pull-streams than I ever did streams1 |
| 20:32:38
| <creationix> | and in nearly 100% of my onRead callbacks, the first thing I do is two if statements breaking out error and end events to their own handlers |
| 20:33:29
| <dominictarr> | ah, in pull-streams you can handle err and end exactly the same in most cases! |
| 20:33:39
| <dominictarr> | it just breaks fibers |
| 20:34:01
| * timoxley | quit (Ping timeout: 256 seconds) |
| 20:34:05
| <creationix> | I think you're writing more middleware and I'm writing more sources and consumers |
| 20:34:23
| <dominictarr> | that is quite possible |
| 20:34:24
| <creationix> | in consumers an end event is very different than an error |
| 20:34:35
| <creationix> | but to something in the middle, they are both just a value to propigate |
| 20:34:45
| <creationix> | most middle pieces don't care |
| 20:34:52
| <dominictarr> | yes, the last value |
| 20:35:18
| <creationix> | that was why in simple-streams, both end and error are encoded with value being undefined |
| 20:35:35
| <creationix> | if (value === undefined) return callback(err); |
| 20:35:42
| <dominictarr> | so, I think the real test is how easy it is to implement things like cat - which combines N streams |
| 20:35:56
| <creationix> | or how hard it is to implement git ;) |
| 20:36:21
| <dominictarr> | well, that is a more difficult comparison |
| 20:36:21
| <creationix> | or more specifically git fetch |
| 20:36:31
| <creationix> | network protocols, multiplexing and embedded substreams |
| 20:36:48
| <dominictarr> | you can compare that stuff if it's separate modules |
| 20:37:09
| <creationix> | so cat should be easy with this spec |
| 20:37:12
| * creationix | gists |
| 20:39:15
| <creationix> | dominictarr, how should cat behave when there is an error in a source? |
| 20:39:19
| <creationix> | should it kill everything? |
| 20:39:57
| <dominictarr> | yes |
| 20:40:33
| <dominictarr> | it should abort all the streams |
| 20:40:43
| <dominictarr> | and end the consumer |
| 20:41:13
| <dominictarr> | but when a stream ends, it should start reading from the next stream |
| 20:41:50
| <creationix> | hmm, cat is a fun case |
| 20:41:53
| <creationix> | good idea |
| 20:42:48
| <dominictarr> | another one is merging n streams, so that you always read the lowest value |
| 20:42:52
| <dominictarr> | like, in a merge sort |
| 20:43:13
| <creationix> | so a cat that also merges? |
| 20:43:27
| <creationix> | I basically just did that for git history |
| 20:43:35
| <creationix> | since history isn't linear |
| 20:43:45
| <dominictarr> | so, cat reads the entire first stream, then the entire second, and so on |
| 20:43:45
| * chilts | quit (Ping timeout: 245 seconds) |
| 20:43:56
| <creationix> | oh, I was doing cat wrong |
| 20:43:58
| <dominictarr> | but merge reads from every stream, and emits the lowest value |
| 20:44:00
| <creationix> | good point |
| 20:44:22
| <dominictarr> | but keeps even pressure on every stream |
| 20:44:40
| <dominictarr> | i.e. minimum in the buffers |
| 20:44:45
| <creationix> | well cat just reads the streams one at a time right? |
| 20:44:51
| <dominictarr> | yes, |
| 20:45:00
| * chilts | joined |
| 20:45:09
| <dominictarr> | . |
| 20:45:11
| <creationix> | yeah, cat is much easier than merge |
| 20:46:23
| <dominictarr> | yeah, my cat is 40 lines, but my merge is 80 |
| 20:48:02
| * defunctzombie | changed nick to defunctzombie_zz |
| 20:49:15
| <dominictarr> | another hard one is to do a binary search on a sorted table, for a range query |
| 20:49:26
| <dominictarr> | that took me 100 lines |
| 20:49:48
| <creationix> | I need to go somewhere in 10 minutes, but hopefully I'll have cat done first |
| 20:49:51
| <creationix> | these are good tests |
| 20:49:59
| <creationix> | much quicker than wasting weeks rewriting jsgit |
| 20:51:02
| <dominictarr> | indeed! |
| 20:51:20
| * defunctzombie_zz | changed nick to defunctzombie |
| 20:53:51
| <creationix> | dominictarr, what if they call read again after I emit End |
| 20:53:53
| <creationix> | just ignore them? |
| 20:54:00
| <creationix> | unefined behavior |
| 20:54:29
| <dominictarr> | yeah, ignore |
| 20:54:48
| <dominictarr> | maybe, log an error |
| 20:54:49
| <creationix> | ok, I think this is cat |
| 20:54:50
| <creationix> | https://gist.github.com/creationix/d05f75643bb0abdf2f90#file-cat-js |
| 20:56:20
| <creationix> | hmm, missing a line |
| 20:56:47
| <creationix> | there, that's better |
| 20:58:34
| <creationix> | it's weird and different, but otherwise very elegant I thing |
| 20:58:37
| <creationix> | *think |
| 20:58:42
| <creationix> | anyway, time to go |
| 21:08:31
| * fuz__ | changed nick to fuz |
| 21:29:52
| * timoxley | joined |
| 21:34:39
| * timoxley | quit (Ping timeout: 256 seconds) |
| 21:51:24
| <dominictarr> | creationix: looks quite good! |
| 21:51:42
| <dominictarr> | a little confused by all declerations |
| 21:51:47
| <dominictarr> | and no closures |
| 21:53:11
| <dominictarr> | if you call read but it's ended it calls onEnd? |
| 21:55:43
| * AvianFlu | quit (Remote host closed the connection) |
| 21:56:15
| * AvianFlu | joined |
| 22:00:38
| * AvianFlu | quit (Ping timeout: 256 seconds) |
| 22:08:48
| <thlorenz> | dominictarr: you pointed me to hipster to make my cursor reappear (remember at nodeconf)? |
| 22:08:57
| <dominictarr> | oh, yeah |
| 22:09:00
| <thlorenz> | looked through the code, but can't find where that is |
| 22:11:45
| <dominictarr> | hmm, it might just be a char code that I wrote out... |
| 22:13:02
| <thlorenz> | dominictarr: no worries, if you happen to run into it just let me know - I'll search the interwebs too :) |
| 22:13:24
| <dominictarr> | generally, I just searched and randomly tried stuff |
| 22:13:55
| <thlorenz> | dominictarr: this looks useful: https://github.com/TooTallNate/ansi.js/blob/master/lib/ansi.js#L37-L38 |
| 22:16:03
| <dominictarr> | aha, I think I got it from charm: |
| 22:16:05
| <dominictarr> | this.write(encode(visible ? '[?25h' : '[?25l')); |
| 22:16:28
| <dominictarr> | I just have https://github.com/dominictarr/hipster/blob/master/lib/render.js#L199 |
| 22:21:51
| <dominictarr> | creationix: okay, so cat is a case where you require abort |
| 22:22:13
| <dominictarr> | the streams could be open files, or open network sockets |
| 22:23:19
| <dominictarr> | for a short lived program it might not matter, but for a server it does. |
| 22:30:36
| * timoxley | joined |
| 22:31:41
| * yorick | quit (Remote host closed the connection) |
| 22:35:17
| * timoxley | quit (Ping timeout: 256 seconds) |
| 22:51:15
| * dominictarr | quit (Quit: dominictarr) |
| 22:52:01
| * Maciek416 | joined |
| 23:09:00
| <substack> | https://github.com/substack/hacker-deps |
| 23:12:12
| * dominictarr | joined |
| 23:21:35
| * kriskowal | quit (Quit: kriskowal) |
| 23:24:07
| * rxgx | joined |
| 23:25:02
| * tilgovi | joined |
| 23:29:26
| <jesusabdullah> | yeah I ran that |
| 23:30:13
| <substack> | \o/ |
| 23:30:27
| <chrisdickinson> | mine doesn't output anything )`: |
| 23:30:52
| <substack> | chrisdickinson: I think there might be a race condition in it |
| 23:30:56
| <substack> | try running it again |
| 23:30:57
| <jesusabdullah> | yeah mine output for browserify-cdn but not a whole lotta other stuff |
| 23:31:23
| * timoxley | joined |
| 23:35:26
| * timoxley | quit (Ping timeout: 240 seconds) |
| 23:41:24
| <substack> | ok found and fixed the issue with empty results |
| 23:41:31
| <substack> | chrisdickinson, jesusabdullah: ^^^ |
| 23:43:52
| <chrisdickinson> | substack: haha, you're usually a bigger contributor to my projects than I am |
| 23:44:26
| <substack> | yay sharing! |
| 23:44:48
| <chrisdickinson> | :D |
| 23:44:50
| <nexxy> | \o/ |
| 23:55:40
| * fuz | quit (Ping timeout: 256 seconds) |