00:07:22  * mikealjoined
00:19:39  <Raynos>ralphtheninja: npm docs repo name
00:19:42  <Raynos>then follow link in npm
00:20:24  * thatguydan_joined
00:21:35  <ralphtheninja>Raynos: awesome thanks, 'npm docs streams2' opened up the browser straight away
00:21:54  <Raynos>yeah
00:21:57  <Raynos>good modules set a docs field
00:22:00  <Raynos>that points to github
00:22:04  <Raynos>Most of my modules to that
00:22:08  <Raynos>bad modules point to npm.im
00:22:25  <Raynos>oh and npm docs streams2 and streams2 are two different things :D
00:22:47  <Raynos>if you want streaming base classes use read-stream, write-stream and read-write-stream
00:22:52  <Raynos>which are all horrible names
00:23:41  * mikealquit (Quit: Leaving.)
00:23:49  <ralphtheninja>oh so when you said streams2 you meant the new ones in core?
00:24:11  * thatguydanquit (Ping timeout: 255 seconds)
00:24:11  * thatguydan_changed nick to thatguydan
00:25:06  <Raynos>I meant isaacs/node#streams2
00:25:11  <ralphtheninja>I wasn't really after any particular repo, just want to browse things fast in general :)
00:25:14  <ralphtheninja>gotcha
00:27:06  <Raynos>cool
00:27:13  <Raynos>another tip is
00:30:17  <Raynos>piping npm search into xargs npm docs
00:32:05  <Raynos>dominictarr, substack: I just realized that I get annoyed at the amount of plumbing it takes to set up reconnect and pipe together network streams & mux demux but tolerate so much bullshit setting up XHR and REST end points
00:32:35  <dominictarr>there is waaaay more bullshit writing REST
00:32:41  <Raynos>why do I even do REST
00:32:51  <Raynos>why do I even tolerate it
00:32:53  <dominictarr>like you have to write all these requests, and then all these responses
00:32:59  <dominictarr>but it's the same stuff
00:33:26  <Raynos>I was finding this connect function annoying: https://gist.github.com/4116665#L40
00:36:24  <Raynos>but now im writing REST end points
00:36:26  <Raynos>and dear god
00:36:27  <Raynos>._.
00:36:33  <Raynos>\there is no comparison
00:37:00  <mbalho>could there be a module that abstracts duplex piping syntax
00:37:07  <mbalho>npm install luigi
00:37:14  <mbalho>and npm install mario
00:37:14  <Raynos>you cant
00:37:29  <mbalho>peerStream.pipe(peers.createStream()).pipe(peerStream) is 3 function calls
00:37:30  <Raynos>i thoughta bout abstracting it
00:37:37  <Raynos>but those are the network streams
00:37:41  <Raynos>network streams disconnect
00:37:49  <Raynos>and reconnection logic is very application specific
00:37:58  <Raynos>you need to expose streaming interfaces and pipes
00:38:14  <Raynos>although I guess we can have a function called duplex
00:38:18  <Raynos>that does the duplex pattern
00:38:24  <Raynos>duplex(peerStream, peers.createStream())
00:38:35  <Raynos>but why bother
00:38:38  <mbalho>yea
00:38:52  <Raynos>ive tried this before
00:39:13  <Raynos>https://github.com/Raynos/mux-demux-shoe#example-server
00:39:17  <jjjjohnnny>npm install ouroboros
00:39:23  <Raynos>but your baking in how to handle network disconnects :/
00:39:26  <Raynos>and you cant do that
00:39:45  <Raynos>jjjjohnnny: doesnt exist
00:40:58  <jjjjohnnny>maybe i spelled it wrong
00:41:18  <mbalho>Raynos: your joke detector is misaligned
00:41:48  <Raynos>yeah
00:41:54  <Raynos>we should actually have ouroboros
00:42:25  <Raynos>function ourobos(stream) { while(stream.pipe) { stream.pipe(stream) } }
00:42:39  <Raynos>function ourobos(stream) { while(stream.pipe) { stream = stream.pipe(stream) } }
00:42:52  <Raynos>PIPE PARTY
00:43:38  <mbalho>lol
00:44:19  <ralphtheninja>haha
00:44:50  <ralphtheninja>that bot cracks me up
00:50:19  <dominictarr>there *might* be a good way to do absract connecting pipes.
00:50:30  <dominictarr>for a given class of applications
00:50:38  <dominictarr>talk to owenb about this
00:51:05  <dominictarr>he's intent on doing this for socketstream
00:55:34  <Krinkle>eh..
00:55:39  <Krinkle>what's LOUDBOT?
00:56:04  <Krinkle>Ah, I see
00:56:36  <dominictarr>Krinkle, everything it says are things people have actually said, using CAPS
00:56:54  <Krinkle>random response, or linked?
00:56:56  <Krinkle>TEST
00:56:57  <dominictarr>when you speak in caps, it's added to LOUDBOT's database
00:57:04  <mbalho>and loudbot is in many rooms
00:57:12  <mbalho>minimum length of a few words
00:57:14  <dominictarr>it's just random i think
00:57:24  <Krinkle>because the last one seems somewhat related
00:57:32  <mbalho>its random
00:57:36  <Krinkle>k
00:57:43  <Krinkle>maybe I'm seeing things
00:57:44  <Krinkle>:P
00:57:49  <dominictarr>but that doesn't stop it from being amazingly relevant sometimes!
00:58:04  <dominictarr>LOUDBOT IS A FRAUD
00:58:15  <dominictarr>LIES LIES LIES
00:58:22  <Krinkle>wtf
00:58:22  <Krinkle>http://www.frenblog.com/illusion/wp-content/uploads/2006/11/dolphin.jpg
00:58:29  <Krinkle>"seeing things"
00:58:54  <dominictarr>DOLPHINS, SEXY DOLPHINS
00:59:16  <rowbit>Hourly usage stats: [developer: 4, free: 50]
01:00:46  <Krinkle>(cf. Doctor Who)
01:01:23  * _sorensenquit (Ping timeout: 245 seconds)
01:06:33  * Krinklepart ("Leaving...")
01:08:57  * tilgoviquit (Remote host closed the connection)
01:21:51  <jjjjohnnny>substack: can we use your drone tomorrow y/n
01:22:02  <dominictarr>rvagg, maybe we should get rid of the 'batch' event, and turn it into a collection of put,del events
01:22:03  <dominictarr>?
01:22:17  <dominictarr>jjjjohnnny, its arrived?
01:23:12  <rvagg>dominictarr: maybe that's something that could be switched with an option?
01:23:29  <rvagg>{ createIfMissing: true, batchEvent: false }
01:23:51  <dominictarr>options are kinda ugly, could make it a plugin instead.
01:29:51  <Raynos>jjjjohnnny: should I bring food on thursday
01:49:41  * nk109joined
01:50:34  <nk109>hi guys, nice utility. one problem I'm running into - i'm trying to use the tunnel feature, but i'm getting "error adding new user" when i paste my ssh public key into it
01:51:16  <nk109>also, is there an eta for ie10 support?
01:59:16  <rowbit>Hourly usage stats: [developer: 0, free: 90]
02:01:23  <mbalho>isaacs: have you read http://dvcs.w3.org/hg/streams-api/raw-file/tip/Overview.htm
02:14:17  <substack>jjjjohnnny: of course you should use the drone!
02:25:53  <defunctzombie>substack: browserify 2.0 thoughts? any new revelations?
02:29:08  <substack>defunctzombie: I think it makes the most sense of required has an option to attach the source param to the json output
02:29:20  <substack>because otherwise I'd be doing 2 fs.readFile() calls
02:30:17  <defunctzombie>you mean to have an option to required to also include the file contents
02:31:06  <substack>yes because it already reads the files in order to compute the dependency graph
02:32:11  <ralphtheninja>mbalho: the api seems rather awkward
02:32:35  <mbalho>its a w3c proposal made by microsoft hehehe
02:33:13  <ralphtheninja>aye, where's the pipe() ffs? :)
02:33:27  <mbalho>.src= is their pipe
02:34:01  <Raynos>wait what
02:34:05  <Raynos>w3c has a streams api?
02:34:29  <substack>many!
02:34:40  <ralphtheninja>mbalho: aaah ok
02:37:49  <mbalho>no this one is literally called Streams
02:38:00  <mbalho>and it is for evented i/o for xhr and files and video and audio
02:38:57  <Raynos>meh.
02:39:05  <Raynos>im not going to use w3c apis directly
02:39:13  <Raynos>and they shouldnt be hard to wrap in a node api
02:41:57  <ralphtheninja>and you don't want to wait 2 years for it either
02:48:50  * st_lukequit
02:52:06  * ralphtheninjaquit (Quit: leaving)
02:52:36  * ralphtheninjajoined
02:56:46  <ralphtheninja>anyone been using streamify? https://github.com/fent/node-streamify
02:57:15  <mbalho>i tried to get that guy to hang out in irc
02:57:29  <mbalho>dont think he gets it tho
02:59:16  <rowbit>Hourly usage stats: [developer: 5, free: 49]
03:00:00  <mbalho>ralphtheninja: i think the need for that lib goes away with streams2
03:00:39  <ralphtheninja>sounds good
03:08:01  * devaholicjoined
03:26:13  <Raynos>hack time!
03:29:42  <Raynos>ralphtheninja: yeah you dont really need taht
03:30:19  <Raynos>https://github.com/Raynos/eventual-stream#example
03:30:21  <Raynos>I wrote one too!
03:30:42  <Raynos>actually that module is still useful
03:30:44  <Raynos>for bullshit apis
03:31:03  <Raynos>that require you to make an async call before you get an object with a createStream function
03:31:43  <Raynos>mongodb for example
03:33:15  <ralphtheninja>I don't want to use bullshit apis :)
03:34:11  <ralphtheninja>lol
03:40:15  * tphummelquit (Quit: tphummel)
03:44:24  * thatguydanquit (Ping timeout: 252 seconds)
03:48:23  <dominictarr>rvagg, hey, I notice that write-stream doesn't pass `options` to this._db.put
03:48:38  <dominictarr>LOUDBOT, twitlast
03:48:38  <LOUDBOT>http://twitter.com/LOUDBOT/status/271097764480425984
03:49:01  <rvagg>dominictarr: then fix it, dear Liza
03:53:07  <dominictarr>rvagg, what does writeStream.add do?
03:53:07  <substack>jjjjohnnny: my sources indicate that the copter hasn't arrived yet but these sources may be unreliable
03:53:16  <substack>the dhl tracking page I mean
03:54:04  <rvagg>dominictarr: writeStream is fstream compatible, add() is there purely for that
03:54:11  <rvagg>add(entry)
03:54:15  <dominictarr>right
03:54:41  <rvagg>fstream compatibility is mainly there for my amusement, not because I think people might actually use it
03:55:02  <rvagg>but backing up a db to disk or a tarball is a neat trick that may come in handy
03:55:14  <substack>Raynos: https://trello.com/board/meteor-roadmap/508721606e02bb9d570016ae
03:55:23  <Raynos>see that
03:55:25  <Raynos>its really boring
03:55:32  <Raynos>OH
03:55:41  <Raynos>clone that board
03:55:48  <Raynos>then start finishign them off one by one
03:55:55  <Raynos>call it the "meteor is slow. I can kick their ass" competition
03:56:17  <Raynos>The only thing interesting is the DDP spec
03:56:52  <substack>Raynos: no most of these features aren't worth duplicating
03:56:59  <substack>don't even bother with databases
03:57:02  <Raynos>agreed
03:57:27  <substack>just plug into dominictarr's serialization engine for scuttlebutt
03:57:56  <Raynos>well I want to see what their DDP spec says
03:58:02  <Raynos>and see whether it's anything of value
03:58:08  <Raynos>mainly interested in latency compensation
03:58:12  <Raynos>but I might just fuck that
03:58:14  * xaqjoined
03:58:18  <Raynos>and write a cache in front of leveldb
03:58:19  <substack>and browserify 2.0 will make it so you can write plugins to load html files into your client-side bundles
03:58:21  <Raynos>and call it latency compensation
03:58:24  <substack>that solves the "templating" issue
03:58:32  <Raynos>substack: and SVG files and CSS files
03:58:34  <dominictarr>rvagg, ah, that is pretty cool. tarball would be a good way to swapout a db.
03:58:35  <substack>yes
03:58:40  <substack>whichever things you need
03:59:16  <rowbit>Hourly usage stats: [developer: 0, free: 60]
04:00:06  <dominictarr>haha "SQL support"
04:00:16  <rvagg>dominictarr: it also makes a great functional test to see that the streams are working
04:00:28  <substack>Raynos: notice OT on the top right
04:00:31  <substack>oh man
04:00:33  <Raynos>lol
04:00:34  <substack>they have no idea
04:00:41  <Raynos>the thing is
04:00:44  <substack>how shitty OT is
04:00:51  <Raynos>even though meteor is full of useless features
04:00:54  <dominictarr>I met one of their developers at cascadia JS
04:00:56  <Raynos>they have the simplest API for real time apps
04:01:07  <Raynos>if you can get an api thats that simple and isn't rotten under the hood
04:01:19  <dominictarr>he was also the author of etherpad
04:01:20  <Raynos>dominictarr: I hope they are intelligent guys
04:01:20  <substack>dominictarr: yep I met somebody working on meteor at realtimeconf to
04:01:22  <substack>*too
04:02:00  <dominictarr>some decisions they have made don't vibe with us
04:02:07  <dominictarr>but where not their target market.
04:02:25  <substack>I wouldn't use socketstream either but I like the direction it has taken lately
04:02:31  <substack>perhaps we can convince meteor likewise
04:02:37  <Raynos>well
04:02:42  <Raynos>if we can get meteor to at least use npm
04:02:49  <Raynos>and maybe apply the fibers pressure
04:02:50  <substack>^ most important thing
04:02:53  <Raynos>just those two alone.
04:02:54  <dominictarr>yeah, socketstream is still got some work to do
04:03:04  <dominictarr>it's a bit nutty in places
04:03:08  <dominictarr>gotta email owen
04:03:51  <ralphtheninja>I've come to the right place, I like this channel :) 5 am here now so have to get some sleep, catch you guys later
04:04:22  <dominictarr>ralphtheninja, catch you later
04:05:14  <dominictarr>meteor appeals to someone who just wants to build something useful, not people who are interested in building systems
04:05:35  <dominictarr>they are going for like, a VB sort of vibe.
04:06:02  <AvianFlu>meteor is an exceptionally foolish setup
04:06:07  * AvianFluis an anti-fan of fibers
04:06:24  * AvianFludrinks more and shakes his fist
04:06:45  * ralphtheninjaquit (Read error: Operation timed out)
04:07:39  <rvagg>dominictarr: i hear he's de-coffeescripting socketstream and trying to make it more node-stream sensible
04:07:56  <dominictarr>that is correct
04:08:02  <dominictarr>he's also using mux-demux!
04:08:56  <rvagg>AvianFlu: http://1.bp.blogspot.com/_n5b21dogdsU/TAU3YfTA_eI/AAAAAAAAALY/v3mkEgh8fC4/s1600/153456pw150.jpg
04:10:09  <Raynos>dominictarr: it's aimed at making real time apps as easy as putting some jquery on your side to make it wish wash
04:10:36  <dominictarr>it doesn't really compare to jQuery
04:10:52  <dominictarr>because it's a monolithic framework
04:11:17  <dominictarr>jQuery _nearly_ fits the one-thing-well philosophy
04:11:28  <Raynos>this cascadia meteor talk is shit
04:11:32  * shykeschanged nick to zz_shykes
04:11:35  <Raynos>jQuery is a monolithic framework
04:11:37  <Raynos>dom traversal
04:11:39  <Raynos>query selectors
04:11:41  <Raynos>and animation
04:11:43  <Raynos>have zero business
04:11:46  <Raynos>being in the same module
04:12:08  <dominictarr>you have to look at jQuery in it's historical context
04:12:15  <Raynos>same with underscore
04:12:18  <Raynos>good tools of the time
04:12:21  <Raynos>tools of the pre-npm age
04:12:25  <dominictarr>yes
04:12:27  <Raynos>PRE NPM AGE
04:12:38  <dominictarr>LOUDBOT, twitlast
04:12:39  <LOUDBOT>http://twitter.com/LOUDBOT/status/271103808145330176
04:12:40  <Raynos>that is now a thing. A thing we use to tell people they need to get with the times
04:13:10  <dominictarr>so this isn't just the javascript age, this is the NPM age.
04:13:33  <Raynos>https://twitter.com/Raynos2/status/271104004665253888
04:13:52  <Raynos>i dont use commonJS because I prefer it over AMD
04:13:52  <dominictarr>lots of people still use jquery
04:14:00  <Raynos>I use it because of NPM
04:14:04  <dominictarr>yeah
04:14:04  <Raynos>lots of people still use .NET
04:14:06  <Raynos>and Java.
04:14:18  <Raynos>also
04:14:25  <Raynos>I was halfway through ranting about meteor's talk
04:14:28  <Raynos>they talk about "pub sub"
04:14:35  <dominictarr>what is that even?
04:14:42  <Raynos>when really they should be talking about "we did it right and build the thing that replicates leveldb"
04:14:53  <Raynos>they talk about how you can publish to keep upto data with a data set
04:15:02  <Raynos>and they send you a stream of SET key value and UNSET key events
04:15:21  <Raynos>which is a shit way to do data replication
04:19:15  <dominictarr>it probably wouldn't work p2p
04:19:21  <substack>how do you even pivot a giant ship like meteor in the right direction?
04:19:56  <substack>it seems like it doesn't know what it should be
04:20:37  <substack>there are some elements of parse-style front-end Meteor.is_{server,browser} noise
04:20:56  <substack>where you just write the frontend and the backend is "taken care of for you"
04:21:08  <dominictarr>firebase also
04:21:09  <Raynos>:D
04:21:18  <substack>but how it lets you write backend code at all is really weird
04:21:23  <Raynos>"pub sub and remote method invocation are the future"
04:21:37  <Raynos>Bullshit.
04:22:06  <Raynos>replicated state and streams are the future
04:22:16  <substack>dominictarr: you should put together a really slick frontend for crdt+scuttlebutt with a video
04:22:21  <substack>I can help with this.
04:22:54  <dominictarr>yeah, we need this for nearForm too.
04:23:10  <dominictarr>the FE stuff is kinda not my scene though
04:23:15  <substack>same
04:23:33  <substack>I get around that by just drawing everything in svg
04:23:36  <dominictarr>I've never actually build FE stuff professionally, except for complete hacks
04:24:03  <dominictarr>Once, I used XSS to fix something on our own site.
04:24:25  <dominictarr>^ that was necessary because we had the north korea of CMS
04:24:35  <AvianFlu>dominictarr: are you working for nearForm now? cause that's pretty awesome
04:24:58  <dominictarr>well, I think "with" is more accurate than "for"
04:25:23  <dominictarr>they are feeding me, but don't tell me what to do.
04:26:32  <dominictarr>I feel "with" indicates a more liberated relationship than "for".
04:27:33  * st_lukejoined
04:27:47  * st_lukequit (Client Quit)
04:28:04  * st_lukejoined
04:28:38  <AvianFlu>dominictarr: I approve!
04:28:45  <AvianFlu>I like those guys
04:28:51  <Raynos>"pieces glue together in an interesting way if you use meteor"
04:28:53  <Raynos>~_~
04:29:02  <AvianFlu>s/interesting/disastrous/
04:29:45  <dominictarr>shoud be: "most of the pieces are already glued together if you use meteor"
04:30:10  <AvianFlu>"like your hand to the side of your face"
04:30:21  <Raynos>:D
04:30:56  <substack>dominictarr: http://michaelochurch.wordpress.com/2012/11/12/the-end-of-management/
04:31:11  <substack>"As far as self-executive people are concerned, their employers are just along for the ride– funding them, and capturing some the byproducts they generate along the way, while they work mostly on their real job: becoming really good at something, and advancing their career."
04:31:59  <dominictarr>we need to expand the ways that people can be unmanaged, but still be fed.
04:32:01  <Raynos>"a chance for us to build a common protocol. Use DDP. it works great with meteor"
04:34:43  * AvianFluquit (Remote host closed the connection)
04:35:13  <st_luke>great article
04:35:40  <st_luke>super accurate about reducing risk
04:36:23  * ITproquit (Ping timeout: 276 seconds)
04:45:57  <dominictarr>relocating
04:46:05  <Raynos>dominictarr: https://github.com/dominictarr/reconnect/pull/5 o/
04:50:36  * dominictarrquit (Ping timeout: 255 seconds)
04:56:14  * st_lukequit (Ping timeout: 240 seconds)
04:59:16  <rowbit>Hourly usage stats: [developer: 2, free: 34]
05:00:51  * xaqquit (Remote host closed the connection)
05:01:07  <jjjjohnnny>substack: yes it has arrived
05:02:26  <jjjjohnnny>Raynos: yes you should bring a dessert from some delicious SF establishment
05:02:43  <Raynos>I see
05:02:47  <jjjjohnnny>alternatively alcoholic or non alcoholic beverage
05:03:07  <jjjjohnnny>or a combination of the two in a bottle of St. Germain (or similar)
05:03:26  <jjjjohnnny>thats a combination ddessert + alcoholic beverage
05:04:17  <substack>hooray!
05:07:21  <Raynos>o/
05:26:43  * mikealjoined
05:32:40  <fotoverite>\o
05:32:51  <fotoverite>I don't get meteor's appeal
05:33:17  <fotoverite>Like at all it's like rails only even more magical. Like an evil unicorn intent on corrupting every good and pure in the world.
05:35:44  <Raynos>well its used
05:35:49  <Raynos>for pooping out real time apps
05:35:50  <Raynos>real easy
05:35:57  <Raynos>like jquery is used for pooping out websites real easy
05:36:14  <fotoverite>Jquery is not really good for any real app work
05:36:26  <fotoverite>I mean some basic sizzle work alright but once you beyond that it's not easy
05:36:41  <ik>waaaooooooooooaaoo
05:36:52  <fotoverite>And meteor seems so insecure as to have no real reason to be used by anyone ever.
05:37:38  <fotoverite>Jquery is easy to manipulate dom I would give you that. Still is a goto for myself when I don;t need to worry about proformance
05:37:40  <Raynos>insecure?
05:37:48  <Raynos>you know whats insecure
05:37:51  <Raynos>replicated indexeddb
05:37:53  <fotoverite>Didn't they only get admin features recently?
05:37:55  <Raynos>peer to peer in the browser
05:38:27  <Raynos>Seriously though
05:38:28  <rvagg>fotoverite: you want dom juice? https://github.com/rvagg/traversty + https://github.com/ded/bonzo ftw
05:38:31  <Raynos>security is done at the network layer
05:38:57  <Raynos>rvagg: l2dom directly
05:38:59  <Raynos>close to the metal
05:39:34  <rvagg>dom traversal is nasty even in new browsers
05:39:40  <Raynos>not really
05:39:43  <rvagg>traversty is ~4k, makes traversal a dream
05:39:47  <Raynos>It's verbose yes
05:40:35  <Raynos>that's all really.
05:40:44  <Raynos>besides who does manual dom traversing :/
05:40:48  <Raynos>i dont do any
05:40:50  <rvagg>bonzo's dom manipulation utils are brilliant now too, all the create/append/prepend/etc. you expect from jQuery
05:41:13  <Raynos>meh
05:41:20  <Raynos>https://github.com/Raynos/insert#example
05:41:24  <rvagg>Y U NO IMPRESSED?
05:41:27  <fotoverite>Why not just use query then besides smaller speed
05:41:36  <Raynos>https://github.com/Raynos/class-list#example
05:41:47  <Raynos>https://github.com/Colingo/data-set#example
05:41:49  <Raynos>One thing. Do it well
05:41:52  <Raynos>not kitchen sink
05:42:04  <Raynos>i dont give a shit about your kitchen sink being only 4kb
05:42:09  <fotoverite>There is a point to monoliths also.
05:42:16  <fotoverite>If you need this set all the time wrap it up
05:42:29  <rvagg>traversty isn't a kitchen sink, it's traversal only, bonzo is a bit of a kitchen sink but we're splitting it up
05:42:29  <Raynos>monoliths are junk left over from the pre-npm age!
05:42:43  <Raynos>Fucking l2npm
05:42:58  <fotoverite>You saw me give a whole talk on why monoliths in general are bad. But there are no absolutes
05:43:02  <fotoverite>absolutes are for fundies
05:43:06  <Raynos>rvagg: it has 20 methods and you call it not a kitchen sink
05:43:11  <fotoverite>we're smarter then that
05:43:21  <fotoverite>Not point in infighting, there isn't a true way.
05:43:23  <Raynos>I just like raging
05:43:26  <fotoverite>Only finding a better middle path.
05:43:28  <Raynos>about anythign really
05:43:34  <fotoverite>I've noticed. :P
05:44:02  <Raynos>rvagg: a) selectors and dom traversal baked into one module. fail. modularize it out
05:44:24  <rvagg>Raynos: selectors are not baked in, you have to plug in a selector engine
05:44:30  <Raynos>b) traversal and "nodelist are arrays" utilities into one module. fail. modularize it out
05:44:32  <rvagg>if you even want one
05:45:11  <Raynos>What i see are three modules
05:45:36  <rvagg>nodes as an array is such a simple thing that modularising it is just silly
05:45:50  <Raynos>and its 450 lines of code :/
05:45:52  <Raynos>way too large
05:46:04  <Raynos>rvagg: https://github.com/Raynos/nodecomposite
05:46:06  <Raynos>do one thing well
05:46:39  <Raynos>node composite is an array of elements treated as a single element on which you can do both batch operations and array manipulation
05:46:45  <Raynos>actually
05:46:47  <rvagg>no, do useful things well, together where it makes sense
05:46:49  <Raynos>node composite is a kitchen sink
05:46:51  <Raynos>that should die!
05:46:57  <ik>i like sinks
05:47:00  <Raynos>it does far too much
05:47:03  <ik>you can clean things in them
05:47:06  <ik>you can also wash things in them
05:47:08  <Raynos></rant>
05:47:10  <ik>and you can get drinking water from them
05:47:14  <ik>and additional things!
05:47:30  <ik>since you need all of those things already you might as well have it in one thing
05:47:45  <rvagg>this MODULE-ALL-THE-THINGS ideology can be pushed a bit too far
05:47:49  <ik>I never woke up one day and thought "boy i sure do like washing things but i am super bummed about the availability of tap water"
05:47:53  <ik>"wish i could remove that part"
05:48:45  <Raynos>rvagg: not really
05:48:57  <Raynos>you know your doing it right when people say your module is too small
05:49:03  <ik>Raynos: you're
05:49:07  <Raynos>https://github.com/Raynos/not/issues/1
05:49:09  <ik>and extremism is a super dumb idea
05:49:13  <ik>so don't do that
05:49:42  <rvagg>Raynos: you know you're doing it right.. because of 'faith' in your ideology? what proof do you have for this 'rightness'?
05:50:06  <rvagg>pragmatism ftw, do what makes sense & works, use patterns and rules where they make sense but don't be slave to them
05:50:08  <rvagg>just get stuff done!
05:50:11  <ik>one time i decided i was right about a thing and that the more people whined about it the righter i was
05:50:12  <Raynos>big stuff is a pain in the ass
05:50:17  <ik>then seven people died and i had to leave the country
05:50:17  <Raynos>stuff only gets bigger and more complex
05:50:21  <ik>so it was a bad day
05:50:22  <Raynos>small stuff gets more complex slower
05:50:38  <fotoverite>Sometimes you need complexity. Look at Ai
05:50:41  <Raynos>rvagg: do whatever works for you
05:50:47  * mikealquit (Quit: Leaving.)
05:50:49  <fotoverite>Not everything will be simple.
05:50:59  <rvagg>priority #1 BUILD COOL STUFF
05:51:05  <Raynos>agredd
05:51:09  <Raynos>that's what im doing ~_~
05:51:17  <ik> /|\
05:51:18  <ik> / \
05:51:19  <ik>WIZARD MODE
05:51:20  <Raynos>i want to replicate leveldb over peer to peer connections
05:53:08  * mikealjoined
05:59:16  <rowbit>Hourly usage stats: [developer: 0, free: 38]
05:59:54  <Raynos>oh a DI for JS
06:04:35  * mikealquit (Quit: Leaving.)
06:04:56  * dominictarrjoined
06:17:43  <Raynos>dominictarr: thanks for making me mimic net with this api
06:17:58  <dominictarr>no problem
06:18:41  <Raynos>because now I can write network topologies
06:19:32  <dominictarr>VERY GOOD
06:19:59  <dominictarr>what sort of topologies will you write?
06:20:23  <Raynos>first one is clique
06:20:26  <Raynos>next is star
06:20:28  <Raynos>then a tree
06:20:38  <Raynos>after that I have to think :P
06:20:54  <dominictarr>tree = like a star of stars?
06:21:15  <rvagg>hey Raynos thanks for commenting on splink, it's evolved since the README and is actually working and I'm building a server-side MVC framework on it.. I'll update the public repo soon but I'd like to continue the discussion about DI sometime
06:21:37  <Raynos>dominictarr: tree as in a binary tree
06:22:02  <dominictarr>it could be an n-ary tree, also
06:22:11  <Raynos>yeah doing binary because its easier :P
06:22:22  <dominictarr>okay
06:23:20  <jjjjohnnny>what use would one have for star shaped network topologies?
06:24:47  <mbalho>summoning net demons
06:24:59  <mbalho>DAEMON SPWANER
06:26:13  <Raynos>GOD DAMN COMPLEXITY >:(
06:26:30  <jjjjohnnny>devil's in the API docs
06:26:43  * tphummeljoined
06:28:33  <jjjjohnnny>YOUR NETWORK IS YOUR NET WORTH
06:28:54  <jjjjohnnny>MAYBE I AM A LITTLE BI ASSED
06:29:35  <jjjjohnnny>thats a nice retard loop
06:29:41  <dominictarr>this is an interesting visualization
06:29:42  <dominictarr>http://xkcd.com/195/
06:29:57  <dominictarr>IP addresses mapped on a hilbert curve
06:30:00  <Raynos>https://gist.github.com/4123383
06:30:03  <Raynos>Clique network!
06:30:14  <Raynos>but its far too complex
06:30:30  <Raynos>i dont know how to make it simpler without hiding complexity / loosing flexibility
06:34:14  <Raynos>oh shit that example wotn work
06:35:31  * tphummelquit (Quit: tphummel)
06:50:47  <dominictarr>oh, hang on, so this uses sockjs and not shoe?
06:51:04  <dominictarr>I though it only did that on the server?
06:52:08  <Raynos>it uses sockjs on both sides
06:52:19  <dominictarr>in the new PR
06:52:23  <Raynos>sockjs and shoe are the same thing.
06:52:29  <Raynos>just different implementations
06:52:43  <Raynos>I guess i can just add sock and keep shoe.
06:53:13  <Raynos>and then you can default to shoe or sock in browserify field and people can choose with require("reconnect/sock") vs require("reconnect/shoe")
06:53:27  <dominictarr>yeah, lets do it that way
06:53:43  <Raynos>ok.
06:53:51  <Raynos>you wanna fix it or shall I?
06:53:56  <dominictarr>you
06:54:01  <dominictarr>you started it
06:54:01  <Raynos>:D
06:55:54  <Raynos>done.
06:58:40  <dominictarr>sweet. [email protected]
06:58:45  <dominictarr>0.3.1
06:59:16  <rowbit>Hourly usage stats: [developer: 3, free: 74]
07:00:59  <Raynos>the only annoying thing is that sockjs-stream needs ws on the server
07:01:02  <Raynos>and ws has binary stuff
07:01:15  <Raynos>dont know any good non-binary ws libraries
07:04:14  <Raynos>btw the bad ass thing about peer-connection-shim is that it works on node & browser. true p2p apps with the exact same code!
07:04:31  <Raynos>now just need to port the rest of node to the browser and the entire browser to node
07:10:16  * defunctzombiequit (Remote host closed the connection)
07:11:14  * fotoverite_joined
07:11:15  * fotoveritequit (Read error: Connection reset by peer)
07:11:15  * fotoverite_changed nick to fotoverite
07:18:21  <Raynos>i hate double json
07:18:28  <Raynos>https://gist.github.com/0dc74ab9dfe8523aa896
07:18:37  <Raynos>Getting double json coming down a stream that im piping into mux demux
07:20:14  <dominictarr>there is a solution to that
07:20:23  <dominictarr>make a redis protocol stream
07:20:37  <Raynos>Oh I see
07:20:44  <Raynos>mdm.createStream fails
07:20:50  <Raynos>if you call it before stream.pipe(mdm).pipe(stream)
07:20:58  <Raynos>i.e. mdm doesn't buffer internally
07:21:02  <dominictarr>that is correct.
07:21:03  <Raynos>or at least is not paused by default
07:21:17  <Raynos>it would be nice if it was paused by default :P
07:21:30  <dominictarr>it was too hard to do that when I first wrote it
07:21:45  <dominictarr>would be easy now
07:21:57  <dominictarr>port it to use duplex
07:24:14  <Raynos>what if I port it to streams2 >_>
07:24:36  <Raynos>ugh not worth doing that
07:24:40  <Raynos>too much of a pain in the ass
07:27:09  <Raynos>ugh mux-demux
07:28:18  <Raynos>https://github.com/dominictarr/mux-demux/pull/17
07:29:47  <dominictarr>yup
07:31:16  <dominictarr>[email protected]
07:34:10  <Raynos>im getting a lot of stream is not writable errors >_<
07:34:30  <dominictarr>how come?
07:38:09  <Raynos>not sure
07:38:12  <Raynos>trying to debug
07:38:28  <Raynos>FUCK
07:38:47  <Raynos>no never mind
07:39:19  <Raynos>im obvouisly to writing something that's already ended
07:41:30  <Raynos>https://github.com/isaacs/readable-stream/blob/master/readable.js#L225
07:41:34  <Raynos>its because im using new pipes
07:41:41  <Raynos>which don't listen for end or close on dest and auto unpipe
07:42:08  <Raynos>thus continue writing
07:43:50  <Raynos>https://github.com/isaacs/readable-stream/issues/32 yeah that was it
07:52:21  <dominictarr>right, they listen for finish right?
07:56:21  <Raynos>they should
07:56:27  <Raynos>ugh bugs
07:59:16  <rowbit>Hourly usage stats: [developer: 0, free: 37]
08:02:26  <Raynos>I bet this is a pause bug ~_~
08:04:54  * elliottcablechanged nick to TuckChesta
08:04:59  <dominictarr>I fear that streams2 is gonna end up just as complicated as streams 1
08:05:24  * TuckChestachanged nick to tuck
08:07:39  <Raynos>i fear that too
08:07:44  <Raynos>btw i have this weird bug
08:07:51  <Raynos>where mux-demux gets the header from header-stream
08:07:54  <Raynos>and then crahses :D
08:10:11  <dominictarr>you have to use header-stream on both ends
08:10:33  <Raynos>i do
08:10:44  * defunctzombiejoined
08:10:55  <dominictarr>can you post a runnable script?
08:11:13  <Raynos>with extreme difficulty
08:12:05  <dominictarr>if you can, I'll have a look.
08:14:44  <Raynos>its hard to reproduce
08:14:50  <Raynos>and there is a lot of complexity in this example
08:15:16  * defunctzombiequit (Ping timeout: 248 seconds)
08:15:50  <Raynos>https://github.com/Raynos/peer-connection-pool start example/server.js then start two example/simple.js
08:16:02  <Raynos>sometimes they break
08:16:16  <Raynos>example/simplejs are supposed to connect to each other. Try starting them in arbitary orders
08:16:53  <dominictarr>SWEET, we've got atomic job queues with prehooks now!
08:17:38  <dominictarr>now, I just have to support multiple map reduces in one db.
08:17:45  <rvagg>dominictarr: btw I'm thinking 'maxEntries', 'maxResults', or perhaps even just 'count'.. I'm not sold on 'number'
08:18:08  <Raynos>yeah cant really reproduce that bug on peer-connection-pool so just ignore it
08:18:33  <dominictarr>rvagg, what about "limit"?
08:18:40  <dominictarr>thats what it is in couchdb.
08:22:17  <rvagg>limit makes sense I guess
08:24:18  <dominictarr>wow, I'm at 99 commits on map-reduce already
08:24:29  <fotoverite>nice
08:25:17  <dominictarr>and started working on this on friday.
08:25:37  <dominictarr>didn't touch it sunday, so that is ~25 commits a day
08:59:16  <rowbit>Hourly usage stats: [developer: 1, free: 58]
09:00:37  * tuckchanged nick to elliottcable
09:01:30  <juliangruber>st_luke: are you using jilla? :)
09:02:09  <juliangruber>pkrumins: I bought both shirts!
09:02:31  <juliangruber>Raynos: pingback
09:02:38  <Raynos>hi
09:02:44  <Raynos>i hate webrtc
09:02:44  <juliangruber>hey
09:02:49  <Raynos>its so fucking beastly
09:02:58  <juliangruber>the api looks so big
09:03:04  <juliangruber>didn't yet do anything with it
09:07:46  <substack>juliangruber: hooray!
09:11:45  <substack>I wrote some of the new browserify stuff in a park
09:21:15  <juliangruber>I regularely force myself into writing short code by setting a deadline to the arrival of my 10m metro
09:25:14  <substack>mbalho: another huge xhr problem is how you can only .send() data once
09:25:25  * elliottcablechanged nick to elliotcabk
09:28:31  <Raynos>fuck xhr
09:28:36  <Raynos>open udp webrtc sockets
09:28:44  <Raynos>fuck browsers
09:28:52  <Raynos>open udp sockets directly. fuck security
09:33:55  * elliotcabkchanged nick to elliottcable
09:38:35  * purrquit (Remote host closed the connection)
09:38:43  * purrjoined
09:39:57  <Raynos>BUG FOUND
09:40:13  <Raynos>man that feeling
09:40:14  <Raynos>is good
09:50:30  * devaholicquit (Ping timeout: 276 seconds)
09:59:16  <rowbit>Hourly usage stats: [developer: 0, free: 67]
10:04:14  * ITprojoined
10:22:35  <substack>jjjjohnnny: phase 1 of drone project involves implementing motion detection based on http://gandalf.psych.umn.edu/users/schrater/SchMechVisMotion.pdf
10:23:19  <substack>phase 2 involves reading the ultrasonic proximity sensor stream
10:23:27  <substack>then we can create a point cloud just be flying around
10:23:35  <substack>*by
10:23:56  <Raynos>https://github.com/Raynos/peer-connection-pool#example
10:24:01  <Raynos>finished the pool thing
10:24:11  <Raynos>thats about as low bullshit as the webrtc api can get
10:24:38  <substack>jjjjohnnny: and I can just finish that fft streaming module to implement the band-pass filter necessary for the motion detection algorithms
10:24:49  <substack>Raynos: hooray!
10:24:59  <Raynos>the best thing about it is
10:25:05  <Raynos>that code works on both server & client verbatim
10:25:36  <substack>sweet
10:25:53  <Raynos>the rest is actually building distributed network topologies and doing distributed things :P
10:26:23  <substack>api looks pretty simple
10:26:57  <Raynos>i also host a signal channel at signalchannel.co
10:27:07  <Raynos>so you can just use that and write pure client p2p apps without any servers
10:27:49  <substack>except that the readme depends on a file in example/
10:27:51  <substack>that's a bit odd
10:28:04  <Raynos>yeah i didnt want to inline the implementation of connect
10:28:05  <Raynos>second.
10:32:03  <substack>maybe you could make that connect thing a separate module?
10:34:12  <Raynos>i moved it to signal-channel
10:34:52  <Raynos>https://github.com/Raynos/peer-connection-pool#example
10:34:55  <Raynos>ok so now its cleaner
10:35:20  <Raynos>now I just need to do a "build your own signal channel / peer connection pool" talk
10:36:19  <Raynos>oh and I need to fix all the complexity :P
10:36:20  <Raynos>and write tests
10:36:22  <Raynos>maybe later :D
10:59:16  <rowbit>Hourly usage stats: [developer: 6, free: 51]
11:14:23  * mikealjoined
11:49:13  * ralphtheninjajoined
11:50:50  <ralphtheninja>been dreaming about streams all night, I see streams everywhere
11:59:16  <rowbit>Hourly usage stats: [developer: 8, free: 49]
12:59:16  <rowbit>Hourly usage stats: [developer: 6, free: 65]
13:12:51  * defunctzombiejoined
13:15:43  <substack>defunctzombie: hoy!
13:16:11  <substack>started part of the browserify v2 overhaul using required
13:17:29  * defunctzombiequit (Ping timeout: 256 seconds)
13:18:39  <substack>at kinuta park in tokyo no less
13:27:55  <ralphtheninja>substack: what's required?
13:28:05  * ec_joined
13:28:09  <substack>ralphtheninja: https://github.com/shtylman/node-required
13:28:16  <ralphtheninja>danke
13:28:19  <substack>the module that the browserify v2 plugin system will be based around
13:28:26  <ralphtheninja>ok cool
13:28:52  <substack>https://gist.github.com/4048765
13:34:30  * ecquit (*.net *.split)
13:35:15  <ralphtheninja>hehe this is exactly what I was thinking about yesterday, awesome that someone did it already :)
13:35:38  * ec_changed nick to ec
13:43:50  * sorensenquit (Ping timeout: 246 seconds)
13:44:10  * Madarsquit (Ping timeout: 246 seconds)
13:44:19  * Madarsjoined
13:44:28  <ralphtheninja>I can definitely have use for required
13:44:37  * sorensenjoined
13:46:56  * AvianFlujoined
13:59:17  <rowbit>Hourly usage stats: [developer: 2, free: 75]
14:04:27  * AvianFluquit (Remote host closed the connection)
14:13:34  * defunctzombiejoined
14:18:21  * defunctzombiequit (Ping timeout: 265 seconds)
14:59:16  <rowbit>Hourly usage stats: [developer: 0, free: 105]
15:26:21  * defunctzombiejoined
15:45:51  * yorickjoined
15:52:15  * ralphtheninjaquit (Read error: Operation timed out)
15:59:16  <rowbit>Hourly usage stats: [developer: 1, free: 54]
16:16:59  * _sorensenjoined
16:34:41  * st_lukejoined
16:46:44  <mbalho>aww hell yea http://droneolympics.com/
16:47:06  <mbalho>substack: i gotta get arduino running on your drone before that
16:49:31  <CoverSlide|TPFR>oh schnitt
16:59:16  <rowbit>Daily usage stats: [developer: 84, free: 1405]
16:59:17  <rowbit>Hourly usage stats: [developer: 2, free: 74]
17:00:58  * AvianFlujoined
17:20:19  * saijanai_joined
17:36:41  * CoverSlide|TPFRchanged nick to CoverSlide
17:52:24  * tphummeljoined
17:59:16  <rowbit>Hourly usage stats: [developer: 4, free: 24]
18:00:20  * shuaibjoined
18:10:35  * blakmatrixjoined
18:11:36  * blakmatrixpart
18:29:46  <mbalho>http://secretvolcanobase.org/~tef/distributed_social_networks.txt lulz
18:32:43  <dominictarr>okay, now we can run multiple map reduces.
18:32:54  <defunctzombie>anarchy! http://www.richardrodger.com/2012/11/03/why-i-have-given-up-on-coding-standards/#.UK0dyuOe_XE
18:33:00  <dominictarr>maybe it's time for a readme?
18:33:07  <mbalho>this presentation is funny http://www.mailinator.com/tymaPaulMultithreaded.pdf
18:33:16  <mbalho>● Multithreaded server coding is more
18:33:16  <mbalho>intuitive
18:33:16  <mbalho>– You simply follow the flow of whats going to
18:33:16  <mbalho>happen to one client
18:33:31  <mbalho>two bullet points to settle the debate!
18:33:38  <mbalho>we all better switch architectures!
18:33:54  <fotoverite>Yes lets. Throw away all the things!
18:34:39  <dominictarr>threads are perfect if you only ever do one thing after another
18:35:03  <mbalho>threads are for neckbeards: discuss
18:35:20  <AvianFlu>my neck is pretty scruffy, and I'm not so much a fan
18:35:21  <AvianFlu>so there's that
18:36:23  <dominictarr>the problem with threads, is that they have no real world metaphor.
18:36:29  <dominictarr>what the hell are they like?
18:36:51  <mbalho>theyre like a flock of flying street food stalls in thailand operated by pigeons
18:36:54  <fotoverite>Who the hell cares about real world metaphores
18:37:22  <fotoverite>I thought they were like a bunch of sparrows carrying a bowling ball.
18:37:45  <jjjjohnnny>substack: aha a point cloud to know proximity
18:38:03  <st_luke>I am clean shaven and I don't much care for them
18:38:04  <mbalho>its an octopus pizza chef throwing 19 pizza doughs on a unicycle
18:38:58  <jjjjohnnny>whats that state of open source face recognition
18:39:07  <mbalho>opencv
18:39:27  <mbalho>c++ lib with node bindings
18:39:41  <mbalho>https://github.com/peterbraden/node-opencv#face-detection
18:40:02  <dominictarr>fotoverite, if you have realworld metaphores, then you can understand them easier
18:40:06  <dominictarr>and explain them.
18:40:23  <jjjjohnnny>mbalho: has you seen this in action?
18:40:27  <dominictarr>all good ideas have real worth metaphores.
18:40:38  <st_luke>why can't I hold all these limes
18:40:39  <fotoverite>Yes, but computers in general are hard to get. This is why we lie constantly and simplify
18:40:56  <fotoverite>Get another set of hands st_luke
18:41:12  <AvianFlu>st_luke: learn to juggle
18:41:14  <AvianFlu>I recommend it highly
18:41:19  <dominictarr>so, if threads are like anything
18:41:49  <jjjjohnnny>threads are like looms?
18:41:58  <fotoverite>We make doctors out of them?
18:42:08  <dominictarr>they are like having tunnle vision, with no ability to multitask, but having a cloning machine
18:42:25  <dominictarr>so, if you need to do two things at once, then you just clone your self
18:42:45  <st_luke>dominictarr: mosh is changing my life
18:42:49  <st_luke>I wish I used it sooner
18:42:52  <dominictarr>st_luke, very good
18:42:54  <fotoverite>I need to use mosh myself it looks good
18:43:18  <dominictarr>while event io, is like having cell phones
18:43:37  <mbalho>ooh i like that
18:43:44  <dominictarr>you order a pizza, and then watch tv, whatever.
18:44:20  <dominictarr>then when the pizza arrives (heaps of stuff had to happen for this to occur, but you don't want to know the details) the pizza guy calls you
18:44:22  <mbalho>this narrative will work well in node for cats
18:44:31  <mbalho>cats chillin on the couch eating cat food
18:44:43  <dominictarr>and you stop watching tv for a second and pay for the pizza
18:45:07  <dominictarr>and cats with phones
18:46:57  <st_luke>but cats probably don't care about phones
18:47:19  <dominictarr>the cloner machine is deffinately a metaphor, but I don't think it's real world
18:47:22  <mbalho>they do if it enables them to be lazier
18:47:28  <dominictarr>haha
18:47:48  <dominictarr>that is why they are learning javascript afterall.
18:48:10  <mbalho>guize the secret is that really js for cats is targeted at lazy americans
18:48:13  <mbalho>dont tell anyone though
18:48:16  * mbalhosecrets
18:48:38  <jjjjohnnny>most non digital machines are a a single threaded thing
18:49:00  <jjjjohnnny>the assambly line is a series of threads
18:49:28  <mbalho>threads are like sweat shops
18:49:47  <mbalho>Nike Threadz
18:51:44  <CoverSlide>Threads are like a snowmobile racing across the tundra and then suddenly it flips over, pinning you underneath. At night, the ice weasels come.
18:53:27  * mbalhochanged nick to gylertillies
18:53:41  * gylertillieschanged nick to mbalho
18:53:54  <st_luke>threads are like when policemen pepper spray nonviolent protestors
18:54:09  * ralphtheninjajoined
18:56:10  <jjjjohnnny>does not every computer process use a thread?
18:56:22  <jjjjohnnny>node uses a single thread
18:56:42  <jjjjohnnny>they are not evil they just are
18:58:03  <jjjjohnnny>i dont know if i am rite am i rite?
18:59:16  <rowbit>Hourly usage stats: [developer: 9, free: 51]
19:05:38  <jjjjohnnny>In computer science, a thread of execution is the smallest sequence of programmed instructions that can be managed independently by an operating system scheduler. A thread is a light-weight process.
19:06:23  <ralphtheninja>everything has a trade off :)
19:08:34  <jjjjohnnny>its not about the thread its about how you use them in your process
19:09:15  <ralphtheninja>threads can indeed be a good choice, but they often create unforseen problems, it's difficult to do it correct
19:09:33  <jjjjohnnny>no you dont have a choice you are using threads
19:09:38  <jjjjohnnny>full stop
19:11:07  <AvianFlu>you're either managing them yourself, only using one, or using easy-mode behind-the-scenes worker threads like in node (fs & stuff)
19:11:14  <AvianFlu>but there's always threads
19:11:15  <AvianFlu>is what he's saying
19:14:57  <ralphtheninja>sure, every process has at least one thread, the main thread :)
19:15:30  <jjjjohnnny>all of nodejs's asyncronicity and non blocking-ness happen as an abstraction inside the thread the process is using
19:15:48  <Raynos>jjjjohnnny: https://github.com/Raynos/peer-connection-pool#example
19:16:14  <AvianFlu>every node process has 5 threads, the main and 4 workers
19:16:20  <AvianFlu>you just don't need to deal with the workers yourself
19:16:24  <AvianFlu>cause callbacks 'n stuff
19:17:53  <dominictarr>mbalho, the best point that txt file makes is "if it's not decentralized enough to be used for piracy then it's not decentralized enough"
19:19:07  <ralphtheninja>dominictarr, which txt file? been offline for a while
19:19:42  <dominictarr>http://secretvolcanobase.org/~tef/distributed_social_networks.txt
19:20:31  <dominictarr>jjjjohnnny, ralphtheninja st_luke the real argument is between event loops are shared memory threads
19:20:47  <dominictarr>processes are basically threads, but they share nothing,
19:21:04  <jjjjohnnny>Raynos: what am i looking at here
19:21:05  <dominictarr>So you can imagine them as totally separate entities
19:21:22  <CoverSlide>i learned in kindergarten it's always good to share
19:21:40  <dominictarr>jjjjohnnny, tell raynos to describe his stuff in english, not just link to source.
19:21:44  <Raynos>jjjjohnnny: It's abstraction to provide a node-style net API for webrtc
19:21:46  <Raynos>:D
19:22:00  <Raynos>There are event comments in the docs
19:22:04  <Raynos>there is also a one line description!
19:24:46  <ralphtheninja>dominictarr: the tone in that file is very cynical, but has some points
19:25:46  <dominictarr>yeah, adoption is the biggest obstical I think
19:25:53  <ralphtheninja>agree
19:26:20  <dominictarr>and he's right about that having your own data is a ideology not a usecase
19:26:48  <ralphtheninja>there are other successful p2p things, so I don't see why it should fail because being decentralized
19:27:23  <dominictarr>no, but it to get big, it needs to be useful with just a few users.
19:27:27  <dominictarr>and then grow
19:28:26  <st_luke>"If it can't be used for piracy, it isn't decentralized enough."
19:29:04  <ralphtheninja>it also has to be easy to adopt, it must not require installation of some vm or extra software
19:29:06  <fotoverite>Nothing decentralized enough we still need vpns
19:29:16  <ralphtheninja>easy to get started I mean
19:29:52  <jjjjohnnny>Raynos: whats the difference between pool.connect and peers.join?
19:30:51  <jjjjohnnny>there is a ready userbase of encrypted, optionally anonymous p2p
19:31:20  <jjjjohnnny>secrets leakers and journalists
19:31:35  <jjjjohnnny>anarchists
19:31:47  <jjjjohnnny>protestors
19:32:05  <jjjjohnnny>true p2p is not an application, its a protocol
19:32:14  <dominictarr>so, a social network for enemies of the state.
19:33:36  <jjjjohnnny>once there is established a way to post, search, download and decrypt data to a public p2p network, it will get used just like the regular web
19:33:46  <ralphtheninja>at least a social network for people with similar ideas and values that treasure anonymity
19:34:05  <dominictarr>of course, there is TOR
19:34:31  <dominictarr>the people involved need a good way to communicate, and collaborate.
19:34:39  <dominictarr>for it to grow
19:34:39  <ralphtheninja>aye
19:34:45  <jjjjohnnny>social networks are applications, built on top
19:34:51  <ralphtheninja>tor is great, but still too nerdy for the 'average person'
19:35:23  <jjjjohnnny>if the protestors want a social network for staying updated on locations, police activities, etc, that is built ON TOP of the p2p encrypto net
19:35:39  <ralphtheninja>same with e.g. bitcoins, people just don't get it
19:36:05  <jjjjohnnny>what the user doesn't need to know is that their use of the protocol is what make it work
19:36:14  <ralphtheninja>jjjjohnnny: yeah you're right
19:36:25  <jjjjohnnny>because they host fragments of encrypted data
19:36:38  <dominictarr>so, we've gotta get people building distributed systems first
19:37:08  <ralphtheninja>aye, a general protocol for sharing any type of data, and build apps on top of that
19:37:09  <dominictarr>then they will understand how to build p2p
19:37:11  <jjjjohnnny>which may not even be germain to their use of it
19:38:16  <jjjjohnnny>dominictarr: no we need to build the basic protocol for p2p connection and hosting of encrypted data, how to search for it (hashes) and collect the fragments
19:38:57  <jjjjohnnny>an enrypted file becomes a qr code or a new kind of link
19:39:10  <ralphtheninja>sounds like freenet
19:40:41  <jjjjohnnny>its not unlike bit torrent
19:41:11  <dominictarr>also, just figure out how to build applications that work distributed and eventually consistent.
19:41:12  <jjjjohnnny>except that users don't know what file fragments they are hosting
19:42:13  <dominictarr>that is one approach
19:42:30  <jjjjohnnny>true it is only one
19:42:51  <jjjjohnnny>but it goes straight for a p2p hostes network
19:42:52  <jjjjohnnny>hosted
19:42:53  <dominictarr>but if I've learned one thing from observing/using/working on a bunch of crappy systems,
19:43:07  <dominictarr>it's that you are tightly coupled to your persistence
19:43:17  <dominictarr>and you need to choose the right pattern
19:43:56  <dominictarr>it's not something you can successfully abstract away, except for a given class of applications.
19:44:25  <jjjjohnnny>my feeling is that persistence is got through replication
19:44:56  <jjjjohnnny>and that perhaps persistence is not gauranteed after some ammount of time
19:45:25  <jjjjohnnny>like if I post my location to the public p2p cryptograph, it should be available for a while
19:45:36  <ralphtheninja>I dont think that the data have to be encrypted and split up into fragments, might as well just store it unencrypted locally and share with your trusted peers
19:45:56  <jjjjohnnny>ralphtheninja: that is a second layer
19:46:04  * tilgovijoined
19:46:22  * defunctzombiequit
19:46:24  <jjjjohnnny>ralphtheninja: the first layer should be a publicly available network of encryption and anonymous use
19:46:55  * defunctzombiejoined
19:47:17  <jjjjohnnny>which is the protocol
19:47:34  * zz_shykeschanged nick to shykes
19:47:55  <jjjjohnnny>if you want a trust network you can trade public keys or just built your own network using the same protocol
19:47:56  <ralphtheninja>totally with you there
19:48:11  * defunctzombiechanged nick to shtylman
19:49:14  <ralphtheninja>so the first layer is basically just taking care of the infrastructure, how nodes communicate etc?
19:50:03  <jjjjohnnny>the reason i suggest a public net of encrypted fragments (replicated across multiple nodes) is so that people can't be caught hosting files, even tho it necessary that they do for the network to exist
19:51:19  * shtylmanchanged nick to defunctzombie
19:51:24  <jjjjohnnny>ralphtheninja: yes, and how to locate files (similar to bit torrent), upload files, etc
19:51:25  * defunctzombiequit (Client Quit)
19:51:44  * defunctzombiejoined
19:52:38  <jjjjohnnny>from a user perspective, they should get a link, and their cryptograph application should know how to find that file on the public net (if it still exists)
19:53:10  <jjjjohnnny>or whatever the application does for them
19:53:31  * AvianFluquit (Remote host closed the connection)
19:53:51  <jjjjohnnny>Raynos: this is why we need true p2p in node, so that any kind of decentralized p2p app of this kind can be built
19:54:13  <ralphtheninja>yeah it's a good idea, but it will mean an extra overhead, might slow the network down
19:55:06  <jjjjohnnny>ralphtheninja: thats a tradeoff. but if designed well, a popular in a certain area (or among an ad hoc network) should be highly available
19:55:12  <jjjjohnnny>a popular file**
19:56:08  <ralphtheninja>I really like the idea of splitting data into fragments
19:57:21  <ralphtheninja>I'm a p2p n00b really, have too little basic knowledge, but it's fascinating and I think it's the future
19:58:11  <jjjjohnnny>if the user wants to save the file after downloading it from around the network, they can, just as we do in browsers
19:58:56  <jjjjohnnny>but for many applications, the data is time sensitive
19:59:16  <rowbit>Hourly usage stats: [developer: 3, free: 49]
20:00:06  <jjjjohnnny>i have made some naive attampts at tcp p2p but have failed
20:01:16  <jjjjohnnny>STUPID NATS BE BUGGIN
20:01:50  <jjjjohnnny>we need laws that say "NATS HAVE to give you A PUBLIC IPV6 ADDRESS! GRAWRR!!"
20:02:21  <ralphtheninja>should be possible to just gossip between nodes of where data is located, eventually your node knows there the file is
20:02:35  <jjjjohnnny>ok laws wont work but i dont know how else to push that
20:02:37  <ralphtheninja>query through gossip
20:02:56  <ralphtheninja>knows where*
20:03:17  <jjjjohnnny>yeah thats DHT stuff of which I am totally ignorant
20:04:30  <jjjjohnnny>i think dominictarr is hard at work on that
20:05:22  <dominictarr>I'm currently at work on the stuff for more targeted replication
20:05:44  <dominictarr>like replication between your twitter friends
20:06:10  <dominictarr>but this is the layer that massive anon DHT could be constructed effectively
20:06:25  <dominictarr>and also, that useful apps could be built on
20:08:19  <jjjjohnnny>its cool that we can prototype this technology on games and fun apps
20:08:21  <ralphtheninja>however, I think the protocol must be open and tech agnostic, obviously we'd implement it with node, but if others communities want to implement it in another enviroment that should be grand
20:09:00  <mbalho>http://telehash.org/ i know the author of this
20:09:28  <jjjjohnnny>my secret goal is to raise a bunch of money to create games that will push this technology up and out
20:10:02  <mbalho>jjjjohnnny: thats not too far fetched honestly
20:10:04  <st_luke>gonna try out leveldb in a work project
20:10:09  <st_luke>was building couch then I realized I just didn't need that much
20:10:19  <jjjjohnnny>mbalho: yeah i have always wanted to know how to effectively use telehash
20:10:27  <mbalho>well its still kidna vaporware
20:10:34  <mbalho>but i like the protocol
20:10:49  <mbalho>i think we should implement it
20:10:52  <dominictarr>jjjjohnnny, good plan
20:11:02  <jjjjohnnny>mbalho: yeah i mean push the development internally, while we make games for fun and profit
20:12:16  <jjjjohnnny>oh telehash is vaporware. I asked him about it and he kind gave me the look that said it might not be used as advertised, i dunno
20:14:16  <jjjjohnnny>if he wants to come out and tell us how to use it ...
20:14:25  * tomshredsjoined
20:14:40  <mbalho>its not too difficult to understand
20:15:03  <mbalho>there are partial implementations but the reason i say its vaporware is because nobody is really driving development
20:15:06  <mbalho>making it production level
20:17:18  <dominictarr>mbalho, what does it do, basically?
20:17:38  <ralphtheninja>mbalho: implementing it is a great idea
20:18:10  <ralphtheninja>and it's udp as well, rock n roll
20:18:35  <mbalho>http://telehash.org/proto.html
20:19:31  <mbalho>core part is: . Every Switch maintains a list of it's neighbors, some further from that, and a few from very far away
20:20:06  <mbalho>basically it gives you p2p message routing
20:20:57  <dominictarr>right, so it makes it easy to send p2p messages
20:21:05  <dominictarr>it handles the nat busting etc
20:22:12  <ralphtheninja>dominictarr, what's nat busting?
20:22:52  <dominictarr>it's about sending messages direct to machines that don't have their own IP
20:23:16  * TylerGilliesjoined
20:23:20  <dominictarr>because they are behind a router, or firewall
20:23:26  <ralphtheninja>check
20:23:27  <mbalho>i found this article once but have since lost it that explains how most routers open ports on outgoing pings and wait for the response to come back through the same port
20:23:34  <TylerGillies>yo dawg i herd you like browsers
20:23:45  <mbalho>so if you only send an outgoing ping and tell someone else to send data to you back through the same port the router will let it through
20:23:53  <mbalho>TylerGillies: have you implemented nat busting in telehash
20:24:00  <TylerGillies>yeah
20:24:09  <TylerGillies>nat busting is interesting
20:24:19  <TylerGillies>especially on airport routers
20:24:24  <TylerGillies>since they use port randomization
20:24:33  <TylerGillies>so traditional nat doesn't work very well
20:24:38  <TylerGillies>nat busting rather
20:25:02  <mbalho>jeremie miller told me about weird tcp over udp spec
20:25:08  <mbalho>some weird*
20:25:13  <mbalho>from a couple of years ago
20:25:33  <TylerGillies>theres a few things that hack tc over udp
20:25:45  <TylerGillies>theres an article on busting nat with tcp and not needing to use udp
20:26:36  <jjjjohnnny>I followed the supposed methodology of that and failed
20:26:54  <TylerGillies>jjjjohnnny: me too, but i am have airport router so not surprising in my case
20:27:01  <TylerGillies>i have*
20:27:16  <jjjjohnnny>oh by airport you mean wifi?
20:27:35  <TylerGillies>jjjjohnnny: i mean the router, which has wifi ability
20:27:36  <mbalho>http://www.apple.com/airportextreme/
20:28:04  <TylerGillies>they use NAT-PMP
20:28:16  <TylerGillies>which is a proprietary apple protocol
20:28:47  <TylerGillies>i wrote a script for nat punching it in like 3 lines of code though
20:29:11  <dominictarr>interesting
20:29:12  <jjjjohnnny>I could find no correlation between what port i listen on with my local net.Server and the outermost NAT port, though I could predict the latter
20:29:34  <dominictarr>you have a port per socket, right?
20:29:51  <dominictarr>like, outgoing connections get assigned a port?
20:29:56  <mbalho>TylerGillies: we are currently working on distributed systems in node
20:30:00  <mbalho>(generally)
20:30:06  <jjjjohnnny>yes, on your machine, your ruoter, and the ISP NAT itself
20:30:43  <TylerGillies>mbalho: if the network has known peers you could use hamachi and not have to worry about nat
20:30:48  <jjjjohnnny>but when is set my local to listen on a port it has no correlation to the port my NAT is going to set
20:31:02  * shuaibquit (Quit: Textual IRC Client: http://www.textualapp.com/)
20:31:25  <jjjjohnnny>so even tho i can predict that NATs port (which was sequential), i could not poke through down to my local
20:31:29  <TylerGillies>mbalho: or tell people to install hamachi and join a public network, i have one setup at pdxtech with password pdxtech
20:31:54  <mbalho>TylerGillies: thats too nerdy tho
20:32:13  <jjjjohnnny>i'll try UDP later, hopefully it will work as the internet says, cuz it doesn't sound hard. but tcp was a bust.
20:32:35  <TylerGillies>the only way to do good nat is have a list of nat breaking techniques and have the app go through them until it has success
20:32:43  <TylerGillies>thats what most p2p apps do like bit torrent clients
20:33:20  <mbalho>like socket.io but for nat
20:33:29  <TylerGillies>nat.io
20:33:34  <mbalho>nattydread
20:33:38  <mbalho>nattyreed
20:34:45  <ralphtheninja>I like nattydread :)
20:41:30  <TylerGillies>theres also the localtunnel gem
20:41:45  <TylerGillies>which creates a tunnel through localtunnel.com to your localhost
20:42:00  <TylerGillies>i use that for testing stuff that needs oauth
20:42:11  <mbalho>is that jeff lindsays thing
20:42:13  <TylerGillies>actually oauth can use localhost
20:42:29  <TylerGillies>but stuff that requires an actual public thing
20:42:32  <TylerGillies>mbalho: not sure
20:43:11  <mbalho>oh yea jeff just wrote this blog post after i guilt tripped him into it http://progrium.com/blog/2012/11/19/from-webhooks-to-the-evented-web/
20:44:56  <mbalho>guize we need a single page on the internets that has everything we are working on
20:45:21  <mbalho>the way http://www.pocoo.org/ does it
20:47:34  <jjjjohnnny>a publicity site would help the funding effort
20:48:10  <jjjjohnnny>we'll put it on the mad science agenda
20:48:10  <mbalho>mr john gettin all formal PR on us
20:48:33  <jjjjohnnny>I got a suitacase full of faces
20:48:47  <st_luke>do you have any suits
20:49:44  <jjjjohnnny>actually mine are mostmodernist, which means dynamic and changeable :D
20:51:00  <jjjjohnnny>see what is mostmodernist (correct) http://johnnyscript.tumblr.com/post/30260503885/i-used-to-have-a-website-called-mostmodernist
20:51:38  <mbalho>lol
20:53:21  <TylerGillies>im a premodernist, because anyone who considers our society modern is just arrogant
20:53:30  <mbalho>i really wanna get this dude into node http://www.codeofhonor.com/blog/scaling-guild-wars-for-massive-concurrency
20:53:37  <mbalho>he wrote starcraft
20:53:44  <mbalho>and is my childhood hero
20:53:59  <TylerGillies>not sure if correlation or causation
20:54:09  <st_luke>mbalho: he writes awesome blog posts
20:54:11  <st_luke>really fun to read
20:54:12  <mbalho>yea
20:55:08  <st_luke>comes off as a super nice guy as well
20:55:59  <st_luke>maybe that's what happens when people stay away from programming communities
20:56:38  <mbalho>hahah
20:58:45  <ralphtheninja>I haven't met a single douche programmer, perhaps I've been lucky? :)
20:59:16  <rowbit>Hourly usage stats: [developer: 10, free: 48]
20:59:18  <mbalho>stay away from web frameworks and you should be fine
21:00:34  * tomshredsquit (Quit: Linkinus - http://linkinus.com)
21:01:00  <TylerGillies>st_luke: are you saying people in programming communities turn into dicks?
21:01:14  <TylerGillies>ralphtheninja: you need to meet me
21:01:20  <ralphtheninja>hehehe
21:01:21  <TylerGillies>mbalho: can vouche
21:01:40  <ralphtheninja>TylerGillies: I'm not sure if I want to when you put it that way :p
21:01:52  <st_luke>TylerGillies: no, that's not what I'm saying
21:02:27  <chrisdickinson>does anyone know of a CSS selector parser that you can use against arbitrary nested javascript objects?
21:03:25  <TylerGillies>chrisdickinson: i was gonna say that is a dumb question, then realized it was you asking and that you know more about such things than me :)
21:03:41  <chrisdickinson>haha
21:03:44  * ITproquit (Ping timeout: 276 seconds)
21:03:53  <chrisdickinson>i actually prefixed it originally with "maybe a dumb question:" but thought better of it.
21:04:07  <mbalho>chrisdickinson: theres this guy from moscow who wrote one... hold on
21:04:25  <mbalho>npmjs.org/CSSOM
21:04:31  <mbalho>might do what you want
21:05:29  <dominictarr>I've gotta say, I havn't met anyone who i'd consider a huge douche in the nodejs community
21:06:04  <mbalho>thats a good point
21:06:40  <mbalho>the people who use node and are huge douches tend to not participate in community events for whatever reason
21:06:44  <jjjjohnnny>but if you were a rubyhead your opinion would be different (because nodesters would stick their toungue out at you)
21:07:02  <mbalho>i have met many huge rails douches
21:07:32  <st_luke>the node community is too small for people to be jerks without getting called out on their shit
21:08:02  <jjjjohnnny>im amazed that nodejs is still a fringe group
21:08:25  <mbalho>most programming 'communities' are fringe
21:08:40  <jjjjohnnny>even the technology is fringe
21:09:19  <jjjjohnnny>the trajectory is obvious tho
21:09:22  <chrisdickinson>cssom looks close, but what i'm looking for is something like https://gist.github.com/a05f3ce9ef8573094062
21:09:50  <mbalho>chrisdickinson: why not convert your thing to css first
21:09:54  <chrisdickinson>so you can use css selectors to match arbitrary data structures
21:10:11  <mbalho>oh sounds like jsonquery
21:10:11  <jjjjohnnny>and use javascritp to append css
21:10:23  <jjjjohnnny>and write css strings in javascritp
21:10:26  <dominictarr>I am somehow especially good at avoiding interpersonal dramas, I think it might be because I never repeat things one person says to a third person.
21:10:41  <dominictarr>if it was in any way personal
21:11:00  <jjjjohnnny>dominictarr: but... thats what gossis IS
21:11:06  <jjjjohnnny>gossip
21:11:08  <mbalho>dominictarr: your embedded hardware has DRM
21:11:25  <dominictarr>yeah, so I don't do gossip
21:11:35  <chrisdickinson>(you could use that proposed api for the dom, but you could also use it for AST trees)
21:11:39  <chrisdickinson>just double checking before i write it
21:11:48  <mbalho>chrisdickinson: there was a thing that i think was called json query
21:11:52  <mbalho>chrisdickinson: dont remember the exact name
21:12:00  <st_luke>dominictarr: that's a good personal policy
21:12:12  <jjjjohnnny>there is a jsonpath like xpath
21:12:22  <dominictarr>jjjjohnnny, yes several
21:12:41  <jjjjohnnny>i dont use them just sayin
21:13:04  * ITprojoined
21:13:33  <jjjjohnnny>its cool that leveldb keys can be any data structure but I wonder whats the use?
21:13:57  <dominictarr>just use https://github.com/substack/node-pathway
21:14:14  <dominictarr>jjjjohnnny, no, only strings.
21:14:21  <dominictarr>or buffers rather
21:14:52  <rvagg>buffers of anything
21:15:23  <rvagg>dominictarr: I ran into that problem of writeStream() not passing on options to put/batch yesterday, needed a sync:true but had to queue a separate operation at the end just to get it... I think that needs fixing
21:15:50  * defunctzombiequit (Changing host)
21:15:51  * defunctzombiejoined
21:16:01  <dominictarr>rvagg, what does sync: true do?
21:16:14  <dominictarr>does it do the request synchronously?
21:16:22  <rvagg>no, it does a sync fs operation
21:16:37  <rvagg>otherwise it just hands off the data to the OS and the data is written whenever the OS can be bothered
21:17:08  <rvagg>so if you write large chunks of data and then turn around and request them back really quickly, without a sync to force a flush to the fs you won't get it back!
21:17:51  <juliangruber>I thought that couldn't happen
21:17:54  <rvagg>it's much slower to write to the fs but it's a little like guaranteed consistency when you're doing multi-thread access like in node
21:17:57  <juliangruber>leveldb is smarter then that
21:18:09  <rvagg>not if you're accessing via multiple threads juliangruber
21:18:37  <juliangruber>i thought that were only for the case of failure
21:18:44  <juliangruber>and that leveldb keeps things in memory
21:18:46  <juliangruber>hm
21:19:21  <rvagg>if you want to see it in action, you can try it out with this: https://github.com/rvagg/node-levelup/blob/master/test/functional/binary-data-test.js#L27
21:19:30  <dominictarr>rvagg, oh, got it.
21:19:58  <rvagg>comment that out and you won't see all of the data come back out that you just pushed in.. generally only shows up if you're putting in large chunks, small chunks seem to flush quickly enough
21:20:08  <juliangruber>oh ok
21:20:12  <juliangruber>good to know :)
21:20:47  <dominictarr>rvagg, but node only uses a single thread. but you still need sync:true?
21:20:57  <rvagg>it'd be nice to have a flush() actually... but I'm not sure how to do that without writing arbitrary data and then deleting it again--potentially conflicting with existing keys, though I suppose a get() could check first..
21:21:12  <juliangruber>dominictarr a single process?
21:21:16  <dominictarr>yes
21:21:20  <rvagg>dominictarr: no, you'll have multiple worker threads accessing the database for async
21:21:22  <TylerGillies>i should implement the ycombinator in javascript
21:21:33  <TylerGillies>i did it in ruby to get anonymous recursive procs
21:22:00  <juliangruber>rvagg put(_________________________________2)
21:22:02  <juliangruber>or so
21:22:13  <juliangruber>or random with the get
21:22:24  <dominictarr>rvagg, is that because the binding is sync?
21:22:30  <juliangruber>but the use case would have to show where that's necessary
21:22:30  <TylerGillies>http://amix.dk/blog/post/19239
21:22:35  <rvagg>put(+new Date + '____________' + Math.random() + process.pid)
21:22:37  <TylerGillies>someone beat me to it
21:23:02  <juliangruber>dominictarr: multiple js threads can access leveldb at the same time, leveldb is threadsafe
21:23:12  <juliangruber>in most cases
21:23:20  <mbalho>it has a process lock
21:23:25  <mbalho>1 process at a time
21:23:30  <juliangruber>yup
21:23:36  <rvagg>dominictarr: no, the binding is totally async, it doesn't restrict which worker threads can access the leveldb core so it just lets libuv go at it, do many operations at once and you'll have multiple libuv worker threads hammering away
21:23:56  <dominictarr>oh
21:23:58  <dominictarr>okay
21:23:59  <juliangruber>leveldb is designed so well
21:24:09  <dominictarr>what do you need worker threads for though?
21:24:21  <juliangruber>because it's async?
21:24:28  <juliangruber>uv_queue_work
21:24:31  <rvagg>libuv's worker threads
21:24:37  <rvagg>not just one thread
21:25:27  <dominictarr>rvagg, so does a put callback before it's on actually on disk?
21:25:35  <rvagg>yes
21:25:43  * rvaggfinds the docs
21:25:52  <mbalho>juliangruber: were you the guy on twitter the other day who said you wouldnt use the stream api for performance reasons
21:26:03  <rvagg>dominictarr: read about Synchronous Writes at http://leveldb.googlecode.com/svn/trunk/doc/index.html
21:26:26  <rvagg>interesting, the summary is: sync:true = slow but safe, sync:false = fast and generally safe enough
21:26:27  <dominictarr>right, so if I want durability you must set sync:true
21:26:29  <juliangruber>mbalho: yes, in certain cases
21:26:45  <rvagg>dominictarr: you can gate with a single sync:true if you want and all previous writes will be flushed too
21:26:58  <dominictarr>rvagg, okay interesting.
21:27:00  <rvagg>so many sync:false and end with a single sync:true and you'll get more speed
21:27:09  <mbalho>juliangruber: you should still use the api so that it has plumbing with other modules but realize you dont have to emit data in a 1 record:1 emit fashion, you can batch all you want
21:27:29  <dominictarr>I'm hooking into puts, and turning them into batches, and also writing a TODO for the map-reduce jobs,
21:27:35  <rvagg>yay, lets have a streams argument again....
21:27:43  <mbalho>(i missed the first one)
21:27:47  <rvagg>dominictarr: sync:true on a batch is probably a good idea if you care enough about consistency
21:28:01  <mbalho>but not using streams for perf reasons sounds like a misinformed argument to me is all
21:28:12  <dominictarr>so if the process crashes, then you restart the map-reduce and your reduce will be correct
21:28:13  <rvagg>dominictarr: see last para in that Synchronous Writes section in the docs
21:28:16  <juliangruber>dominictarr: https://github.com/juliangruber/node-leveled/blob/middleware/lib/middleware/queue.js
21:28:22  <dominictarr>while still being incremental
21:28:54  <rvagg>sorry, gotta disappear for a while
21:29:17  <juliangruber>mbalho: you mean using a stream just for the api and only emitting one data event?
21:29:51  <dominictarr>juliangruber, I am talking about a durable queue
21:30:09  <dominictarr>it still works across crashes
21:30:29  <mbalho>juliangruber: i mean using the stream for the api and emitting data events as you see fit, if that means only one data event then sure, but it seems like imposing buffering constraints is too rigid
21:30:51  <juliangruber>dominictarrr: I thought you'd do that for performance. A durable queue would be nice
21:30:52  <mbalho>getting data sooner vs paying the inter process cost should be configurable
21:31:28  <juliangruber>mbalho: yeah making it configurable seems the way to go. for the queries i need leveldb to do everyting streaming was way slower
21:31:49  <dominictarr>juliangruber, yes your one looks good for perf
21:32:22  <dominictarr>the great thing about leveldb is that we get to implement all the high level features in js
21:32:37  <dominictarr>so we can easily add whatever we need
21:32:46  <juliangruber>mbalho: I often need like 25.000 rows at once and so far, without any production load, buffering is the way to go
21:33:06  <juliangruber>dominictarr: yes, leveldb is perfect for that
21:33:22  <mbalho>juliangruber: how long does it take for the callback to fire on those 25000 rows and then what do you do with them
21:33:38  <juliangruber>still, things like my range()/find() which are implemented in c make certain things waaaay faster. If you need that
21:33:46  <juliangruber>mbalho: 50ms
21:34:08  <TylerGillies>woo, hackin on nodejs code in my java editor
21:34:49  <juliangruber>mbalho: Push those rows do the client and then send updates
21:35:04  <juliangruber>so I'll implemented a stream for that of course
21:35:45  <mbalho>i guess as long as your query result size < available RAM it doesnt matter
21:36:01  <juliangruber>I just don't want my binding to have much js stuff in it, it should be just c with everything implemented that you need c to do
21:36:09  <mbalho>would be interesting to see concurrency benchmarks with big queries though
21:36:24  <juliangruber>tested with siege and times are constant
21:36:44  <juliangruber>but will have to test with more realistic load levels and see how mem usage etc. go
21:37:05  <ralphtheninja>reading up on streams in node right now, should I stick with 0.8 streams or jump straight onto streams2 like Raynos have? what's your thoughts on this?
21:37:53  <juliangruber>ralphtheninja: check out through by dominictarr
21:38:17  <juliangruber>that helps a lot if just experimenting with one way streams
21:38:58  <juliangruber>and it's api should be quite stable ;)
21:39:26  <dominictarr>juliangruber, mbalho if you had to send the results over the network, then absolutely, you'd stream
21:39:41  <dominictarr>streams are fast because you can go slow
21:39:51  <dominictarr>they are lazy
21:40:02  <ralphtheninja>juliangruber: thanks, but my question is more if I should invest time on 0.8 streams or 0.10
21:40:23  <mbalho>ralphtheninja: its backwards compatible so dont worry too much
21:40:40  <dominictarr>you'll get more help if you do 0.8.
21:40:49  <dominictarr>0.10 is still in development
21:41:09  <dominictarr>is it even rolled into 0.9 yet?
21:41:18  <ralphtheninja>will it be a lot of work converting current 0.8 implementations to the new spec later on?
21:41:27  <ralphtheninja>I dunno, it's on npm though
21:41:51  <ralphtheninja>aah screw it, guess I'll go with current streams for now
21:45:13  <dominictarr>the incidental surface details are different, but their *meaning* is basically the same.
21:45:42  <dominictarr>once you understand what the events and methods *mean* then I don't think it will be too hard.
21:47:59  <Raynos>ralphtheninja; there are bugs in readable-stream
21:48:06  <Raynos>I use 0.10 because I'm nice and help isaacs find the bugs
21:48:10  <Raynos>that's the only reason.
21:52:21  <Raynos>also interop between 0.8 and 0.10 streams is a pain in the ass.
21:52:26  <Raynos>i advice you to just use 0.8
21:59:16  <rowbit>Hourly usage stats: [developer: 6, free: 29]
22:01:59  <juliangruber>docminictarr: been reading conferences.sigcomm.org/co-next/2009/papers/Jacobson.pdf , it might turn into a npm module at the example of scuttlebutt :)
22:02:58  <jesusabdullah>dominictarr: ls-r has no readme :'(
22:03:02  <jesusabdullah>dominictarr: :'(
22:03:22  <jesusabdullah>dominictarr: I don't have TIME to read the source! *indignance*
22:07:49  <fotoverite>Hows Alaska Jesusabdullah?
22:14:25  * shykeschanged nick to zz_shykes
22:19:54  * _sorensenquit (Ping timeout: 240 seconds)
22:30:03  <jesusabdullah>fotoverite: most excellent.
22:30:10  <jesusabdullah>fotoverite: a little chilly of course
22:34:11  <jesusabdullah>fotoverite: was seriously missing the AK coffee scene
22:35:20  * defunctzombiequit (Remote host closed the connection)
22:48:06  * zz_shykeschanged nick to shykes
22:50:24  <substack>jesusabdullah: there's a coffee scene up in towel-keetna?
22:56:13  <fotoverite>Something gotta keep them warm.
22:56:14  * jibayjoined
22:56:30  <fotoverite>I might ned to start a coffee hack group like Max is doing. Would get people interested
22:57:36  <substack>fotoverite: there's already a few NY folks on #nerdtracker
22:57:47  <substack>you could use #nerdtracker + gather to start herding some nerds
22:57:57  <fotoverite>explain to me how nerd tracker works
22:58:27  <jesusabdullah>substack: anchortown
22:59:02  <st_luke>fotoverite: I want to organize some small node meetups in the near future
22:59:14  <st_luke>like ~10 people
22:59:16  <rowbit>Hourly usage stats: [developer: 5, free: 40]
22:59:40  <substack>fotoverite: you can hook foursquare into mbalho's irc bot
23:00:06  <substack>or you can just say "I'm headed to robot coffee"
23:00:28  <substack>and then other folks sometimes show up
23:01:01  <st_luke>substack: is the loudbot source available anywhere that you know of? I want to have it connect to my employer's IRC server to cheer me up from time to time
23:01:35  <substack>st_luke: ask ik
23:01:44  <substack>loudbot got rewritten in erlang recently
23:02:09  <st_luke>substack: that's kind of wonderful
23:02:35  <st_luke>I'm so interested in erlang but I haven't spent any time with it yet
23:05:47  * defunctzombiejoined
23:05:47  * defunctzombiequit (Changing host)
23:05:47  * defunctzombiejoined
23:13:43  * dedis4joined
23:14:21  * defunctzombiequit (Ping timeout: 265 seconds)
23:27:35  * _ritchpart
23:49:35  <jjjjohnnny>hoy shit http://ubuntustudio.org/
23:52:12  * tphummelquit (Quit: tphummel)
23:53:34  <tanepiper>substack: whats your streamy html parser module?
23:59:16  <rowbit>Hourly usage stats: [developer: 3, free: 18]