00:00:01  * ircretaryquit (Remote host closed the connection)
00:00:08  * ircretaryjoined
00:08:00  * dominictarrquit (Ping timeout: 260 seconds)
00:11:44  * No9joined
00:17:49  * blobaumquit (Read error: Operation timed out)
00:19:55  * dominictarrjoined
00:20:17  <defunctzombie>I think a registry that doesn't host any files, but only hosts where/how to get the code and checksums could be interesting
00:20:29  <defunctzombie>it would be simpler too I think
00:21:28  * ryan_stevensquit (Quit: Leaving.)
00:33:57  * thatguydanjoined
00:38:19  * Madarsjoined
00:44:11  * yorickquit (Remote host closed the connection)
00:51:57  * ryan_stevensjoined
00:57:24  * mikolalysenkoquit (Ping timeout: 252 seconds)
01:06:45  * blobaumjoined
01:12:03  * ryan_stevensquit (Quit: Leaving.)
01:20:16  * ryan_stevensjoined
01:30:31  <defunctzombie>isaacs: what does npm do to move the unpacked tarball files from the cache to the final install dir?
01:30:38  <defunctzombie>does it just copy them one by one?
01:30:47  <isaacs>defunctzombie: oh, read through lib/cache.js
01:30:56  <isaacs>defunctzombie: be warned. thar be dragons. it's not prety
01:31:02  <defunctzombie>heh
01:31:24  <defunctzombie>isaacs: rough overview tho, is that what happens once the files are in the 'package' folder on disk?
01:31:30  <defunctzombie>or am I looking for something else?
01:32:00  <isaacs>defunctzombie: it does cache.unpack or something
01:32:02  <isaacs>defunctzombie: yeah
01:32:09  <isaacs>it just dumps from the tarball into the target location
01:32:14  <isaacs>but first it does this whole big stupid dance
01:32:24  <isaacs>which is kind of a nightmare to unravel, but ought to be done at some poine
01:32:25  <isaacs>*point
01:33:02  <defunctzombie>isaacs: so right now I see it unpacks the tarball into a 'package' dir in the cache location
01:33:11  <isaacs>defunctzombie: yes, it does that
01:33:20  <defunctzombie>as well as unpack to final install dir?
01:33:27  <isaacs>cache.add(blah, function(er, data) { .. unpack the cache tarball into the target location .. })
01:33:42  <defunctzombie>I figured after unpacking to cache location, it then would copy those to final target
01:33:53  <defunctzombie>so what is the purpose of unpacking into the cache area?
01:35:34  <isaacs>defunctzombie: i ask myself that regularly.
01:35:42  <isaacs>defunctzombie: afaict, to fuck up shasums?
01:35:55  <defunctzombie>isaacs: hm.. didn't you write it? haha
01:35:58  <isaacs>defunctzombie: it's becuase the code was written before we had node tar and streams.
01:36:12  <isaacs>so there was a value in re-packing all tarballs to remove junk and other stuff.
01:36:17  <defunctzombie>I see
01:36:20  <isaacs>since i couldn't do that in-place like i can now
01:36:23  * ryan_stevenspart
01:36:34  <isaacs>there's a bigass comment in there describing the change that needs to happen.
01:36:37  <isaacs>read through cache.js
01:36:48  <isaacs>once you're about ready to kill yourself, you'll see that comment, and realize just how bad it is.
01:37:28  <defunctzombie>isaacs: so the stuff is unpacked into the cache dir _and_ into target dir
01:37:43  <isaacs>yes.
01:37:46  <defunctzombie>isaacs: is the same true for all tarball urls?
01:37:51  <isaacs>it's true for all urls, period
01:37:59  <isaacs>since registry urls just resolve to a tarball, eventually
01:38:04  <defunctzombie>isaacs: ok, that much I follow
01:38:12  <defunctzombie>isaacs: so then to get from the cache dir to the target dir
01:38:17  <isaacs>and uncompressed tars and single-file packages get packed into a tar.
01:38:23  <isaacs>so everything shows up as a package.tgz file
01:38:40  <defunctzombie>wait.. so if you get some other tar file
01:38:42  <defunctzombie>what happens?
01:38:50  <defunctzombie>it is fetched, untared, then tarred again?
01:38:57  <isaacs>yes.
01:39:01  <defunctzombie>fascinating
01:39:05  <defunctzombie>ok
01:39:06  <isaacs>same for file, folder, everything
01:39:10  <defunctzombie>gotcha
01:39:17  <isaacs>even git urls, though they're a little more optimized now
01:39:33  <defunctzombie>when it is tared, the tarball has a package folder in it
01:39:37  <defunctzombie>where everything is
01:39:39  <isaacs>yes
01:39:40  <defunctzombie>that is what it seems like
01:39:41  <defunctzombie>ok
01:39:51  <defunctzombie>so how would you untar that into the target dir?
01:39:53  <defunctzombie>node tar?
01:39:57  <isaacs>lib/install.js, line 953
01:40:06  <isaacs>function write (target, targetFolder, context, cb_) {
01:40:21  <isaacs>that calls cache.unpack
01:40:36  <isaacs>which first does cache.add if necessary, and then untars into the location, changing the name from "package" to "$name"
01:40:59  <defunctzombie>gotcha
01:41:12  <defunctzombie>what happens if the location already exists tho?
01:41:36  <defunctzombie>would it untar over that? or what
01:41:43  <isaacs>it removes it first
01:41:52  <isaacs>read the code, it's pretty straightforward
01:41:54  <isaacs>once you know where to lok
01:42:33  <isaacs>cache.unpack
01:42:42  <isaacs>lib/cache.js line 1108
01:42:54  <isaacs>(jesus fuck over 1kloc in one file? who the fuck wrote this shit?)
01:42:56  <defunctzombie>ok, so it will always remove it when going down that line of installation/cache work
01:43:09  <defunctzombie>all of these files are a mindfuck
01:44:08  <defunctzombie>isaacs: do you have any documentation on tar?
01:44:28  <defunctzombie>I guess the examples
01:44:30  <defunctzombie>ok
01:45:21  <isaacs>yep
01:49:48  <defunctzombie>this is a mess, tar utils call back into cache to lock things sigh
01:55:33  <defunctzombie>isaacs: what do you use to remove a whole path?
01:55:39  <defunctzombie>do you just shell out to rm -rf?
01:55:48  <isaacs>defunctzombie: rimraf
01:55:49  <defunctzombie>I would think not
01:55:51  <defunctzombie>wut
01:55:57  <isaacs>defunctzombie: http://npm.im/rimraf
01:56:05  <defunctzombie>rofl
01:56:11  <defunctzombie>that is one hell of a name
01:56:33  <isaacs>:)
01:56:37  <defunctzombie>bah.. more graceful fs monkey patch :(
01:56:48  <isaacs>defunctzombie: graceful-fs doesn't monkey patch any more.
01:56:53  <defunctzombie>oh yea?
01:56:55  <defunctzombie>\o/
01:56:56  <isaacs>ya
01:57:05  <defunctzombie>thank god
01:57:20  <isaacs>https://github.com/isaacs/node-graceful-fs/commit/e0073e408b41a158133d9ea25624d45b1a44f2a8
01:59:18  <defunctzombie>nice
01:59:28  <defunctzombie>I was not too happy about the monkey patching
02:01:48  * AvianFluquit (Remote host closed the connection)
02:09:49  * tphummelquit (Quit: tphummel)
02:15:26  * dguttmanquit (Quit: dguttman)
02:16:58  * dguttmanjoined
02:17:35  <defunctzombie>isaacs: npm-registry-client readme example has the word 'latest' after the package name
02:17:53  <defunctzombie>isaacs: what does that actually do if I put something else there?
02:18:00  <defunctzombie>or is that old documentation?
02:18:25  <isaacs>defunctzombie: i dunno. check the code.
02:18:33  <isaacs>i think it just fetches /foo/latest
02:18:37  <isaacs>which would be the "latest" tag of foo
02:18:46  <defunctzombie>isaacs: basically, given the result of npm-reg-client.get('module'), how do you figure out what version to fetch?
02:18:54  <defunctzombie>cause that result just returns all of the versions
02:19:25  <isaacs>defunctzombie: try it. open up your web browser to http://registry.npmjs.org/module/
02:19:28  <isaacs>what do you see?
02:19:30  <isaacs>:)
02:19:55  * dguttman_joined
02:19:57  * dguttman_quit (Client Quit)
02:20:28  * No9quit (Ping timeout: 252 seconds)
02:20:38  <defunctzombie>isaacs: ok, so what are you doing to check semver?
02:20:48  <defunctzombie>assuming I specified something like ~1.0.0
02:20:54  <defunctzombie>you would need to find the appropriate version
02:21:05  <defunctzombie>are you using any specific method there?
02:22:06  * dguttmanquit (Ping timeout: 272 seconds)
02:23:24  <defunctzombie>isaacs: specifically, what sorting is done on the version keys (if there is a special sorting)
02:23:47  <Raynos>apparently im implementing event sourcing on top of mongodb
02:24:43  <Raynos>leveldb people ASSEMBLE
02:25:00  * mikolalysenkojoined
02:25:01  <Raynos>I need a leveldb server I can talk to over TCP
02:26:35  * ralphtheninjaquit (Ping timeout: 255 seconds)
02:27:20  <defunctzombie>Raynos: you need regex
02:27:34  <Raynos>why do I need regex?
02:27:42  <defunctzombie>I dunno
02:27:44  <defunctzombie>but you have a problem
02:27:47  <defunctzombie>and regex is the solution
02:27:50  <Raynos>:D
02:28:06  <Raynos>only if my problem is "HOW I PARSE HTMLZ"
02:28:14  <defunctzombie>hahaha
02:28:26  <dominictarr>Raynos: easy.
02:28:44  <dominictarr>but it does change some assumptions
02:28:50  <dominictarr>because it's not imbedded
02:29:20  <dominictarr>so you need to handle some things, like if two connections want to write to the same key concurrently - how do you handle that?
02:30:13  <dominictarr>how do you want to handle that, I mean.
02:31:45  <Raynos>you assume its impossible
02:31:50  <Raynos>just assert the database is append only
02:31:54  <Raynos>thats good enough for my use-case
02:32:19  <Raynos>i just want an append only log in leveldb
02:32:26  <Raynos>and want to read from it in multiple processes
02:32:49  <dominictarr>okay if you do that it's okay
02:33:05  <Raynos>not sure if leveldb is a good thing for an append only log though
02:33:15  <dominictarr>I think "just never do that" is more accurate thing to say than "assume it's impossible"
02:33:24  <Raynos>Well yes
02:33:29  <Raynos>the server would just allow you to `put`
02:33:47  <dominictarr>yes, and always with a new key
02:33:49  <Raynos>there would be `function add(v) { db.put(monotonicTimestamp(), v)`
02:34:01  <Raynos>i mean just NOT allow you to put
02:34:21  <Raynos>Actually all I need is `add(value)` and `tail(timestamp)`
02:34:27  <dominictarr>so, you could just use a single file for that
02:34:42  <dominictarr>if you are only appending, just have a log file
02:34:46  <Raynos>for some reason the idea of flat files feels dangerous :P
02:35:03  <dominictarr>google calls them sorted-string tables
02:35:10  <Raynos>:3
02:35:19  <dominictarr>if you read through the leveldb literature
02:35:33  <dominictarr>they are like, we use these SSTs all the time
02:35:50  <dominictarr>but we just wanted something a littlebit more general - so: leveldb
02:36:27  <dominictarr>you could use append only, and have a thing to binary search in the file, so you could still do range queries.
02:36:37  <Raynos>sounds like too much work
02:36:54  <Raynos>right now im writing to mongo and offloading the scaling / availability / durability problem to mongo
02:36:55  <dominictarr>binary search is easy!
02:37:45  <Raynos>well yes it is
02:38:42  <dominictarr>you could also keep a partial index, so it would be easy to find a particular value-range.
02:39:02  <dominictarr>since you don't have deletes it the index could be immutable
02:41:42  * AvianFlujoined
02:52:22  * dguttmanjoined
02:55:17  * dguttmanquit (Client Quit)
03:00:20  * dguttmanjoined
03:06:15  * jden_joined
03:06:39  * jden_changed nick to Guest78097
03:10:05  * tilgoviquit (Ping timeout: 252 seconds)
03:10:41  * Guest78097quit (Ping timeout: 255 seconds)
03:23:28  * ins0mniaquit (Ping timeout: 256 seconds)
03:31:08  * mikolalysenkoquit (Ping timeout: 272 seconds)
03:45:21  * dguttmanquit (Quit: dguttman)
03:46:12  * mikolalysenkojoined
03:51:17  * dguttmanjoined
03:59:58  * mikealjoined
04:11:16  * dguttmanquit (Quit: dguttman)
04:14:51  * dguttmanjoined
04:15:43  * dguttmanquit (Client Quit)
04:17:08  * jden_joined
04:17:31  * jden_changed nick to Guest34112
04:22:55  * Guest34112quit (Ping timeout: 256 seconds)
04:24:07  * mikealquit (Quit: Leaving.)
04:26:26  * mikealjoined
04:26:28  * jden__joined
04:34:15  <defunctzombie>isaacs: what if a .tgz from the registry unpacks without a package folder?
04:34:21  <defunctzombie>is that some old style packaging?
04:43:55  * dominictarrquit (Quit: dominictarr)
04:45:22  * blobaumquit (Read error: Operation timed out)
04:56:25  <Raynos>step one done
04:56:36  <Raynos>generic mapReduce from event source to snapshot
04:56:57  <defunctzombie>I have rebuilt npm install haha
04:57:03  <defunctzombie>well, at least the first pieces of it
04:57:40  <defunctzombie>my version also cuts the cold no cache install time of express down from ~6 seconds to ~2 seconds
04:59:09  * defunctzombiechanged nick to defunctzombie_zz
04:59:16  <Raynos>nice
04:59:18  <Raynos>can you rewrite npm
04:59:21  <Raynos>it needs to be done
05:00:02  <Raynos>isaacs: we need npmconf. rewrite npm in one conference
05:08:07  * st_lukejoined
05:35:48  * mikealquit (Quit: Leaving.)
05:36:02  * st_lukequit (Remote host closed the connection)
05:57:32  * mikolalysenkoquit (Ping timeout: 255 seconds)
06:06:06  * mikealjoined
06:06:24  * CryptoQuickjoined
06:18:27  * ralphtheninjajoined
06:24:11  * mikolalysenkojoined
06:24:56  * AvianFluquit (Remote host closed the connection)
06:27:15  <rvagg>Raynos: you around?
06:32:54  * mikolalysenkoquit (Ping timeout: 272 seconds)
06:38:04  * mikolalysenkojoined
06:49:45  * jiang-plusjoined
06:50:18  * mikealquit (Quit: Leaving.)
06:59:16  * CryptoQuickquit (Quit: CryptoQuick)
07:01:41  * mikolalysenkoquit (Ping timeout: 244 seconds)
07:04:49  * st_lukejoined
07:06:05  * mikealjoined
07:17:46  * nk109joined
07:17:47  * nk109quit (Client Quit)
07:20:08  * CryptoQuickjoined
07:29:44  * st_lukequit (Remote host closed the connection)
07:32:07  * shamaquit (Remote host closed the connection)
07:49:15  * jiang-plusquit (Quit: leaving)
08:07:37  * mikolalysenkojoined
08:12:28  * mikolalysenkoquit (Ping timeout: 256 seconds)
09:16:15  * thatguydanquit (Ping timeout: 252 seconds)
09:25:46  * ins0mniajoined
09:29:49  * slaskisjoined
09:31:23  * CryptoQuickquit (Quit: CryptoQuick)
09:58:47  * thatguydanjoined
09:59:02  * thatguydanquit (Remote host closed the connection)
09:59:15  * thatguydanjoined
10:10:20  * defunctzombie_zzchanged nick to defunctzombie
10:13:10  * defunctzombiechanged nick to defunctzombie_zz
10:15:00  * thatguydan_joined
10:15:59  * No9joined
10:17:39  * thatguydanquit (Ping timeout: 260 seconds)
10:17:39  * thatguydan_changed nick to thatguydan
10:19:26  <jesusabdullah>isaacs: ihnfi, I suspect it's a fundamental misunderstanding of how I get the module names and/or how much the module is actually used (the only real use case is load testing npm build systems)
10:53:54  <jesusabdullah>dammit tarr
11:06:46  * dominictarrjoined
11:15:06  * lyaunzbejoined
11:15:39  <jesusabdullah>dominictarr: ^5 OPAN SAURCE
11:16:08  <jesusabdullah>seriously though, good PRs to ecstatic are a great way to wash out the taste of blacklist PRs to hoarders XD
11:16:45  <dominictarr>NO PROBLEM!
11:16:46  <LOUDBOT>THE US HAS LED THE CHARGE ON SERIOUS FINANCIAL SANCTIONS
11:17:46  * lyaunzbe_quit (Ping timeout: 256 seconds)
11:22:54  * thatguydanquit (Remote host closed the connection)
11:23:06  * thatguydanjoined
11:23:09  * thatguydanquit (Client Quit)
11:29:38  * No9quit (Ping timeout: 255 seconds)
11:32:08  * dominictarrquit (Ping timeout: 256 seconds)
11:42:47  * No9joined
11:46:10  * heathquit (Remote host closed the connection)
12:18:20  * fotoveritequit (Ping timeout: 260 seconds)
12:43:13  * heathjoined
12:54:18  * _ddgbotjoined
13:07:49  <jesusabdullah>!recipe falafel
13:07:50  <_ddgbot>http://allrecipes.com/search/default.aspx?qt=k&rt=r&wt=falafell&pqt=k&ms=0&fo=0
13:28:40  * No9quit (Ping timeout: 244 seconds)
13:40:18  * robertjd__quit (Ping timeout: 264 seconds)
13:40:39  * robertjd_joined
13:49:11  <jesusabdullah>seriously thinking about writing an error normalization library called your-errors-are-bad-and-you-should-feel-bad
13:52:13  * yorickjoined
13:52:13  * yorickquit (Changing host)
13:52:13  * yorickjoined
14:01:37  * defunctzombie_zzchanged nick to defunctzombie
14:33:57  * yorickquit (Remote host closed the connection)
14:40:30  * mikolalysenkojoined
15:03:13  * fotoveritejoined
15:16:05  * dominictarrjoined
15:17:22  * yorickjoined
15:17:22  * yorickquit (Changing host)
15:17:22  * yorickjoined
15:22:41  * dominictarrquit (Quit: dominictarr)
15:47:01  * pkruminspart
15:53:23  * AvianFlujoined
16:39:55  * dominictarrjoined
16:51:20  * defunctzombiechanged nick to defunctzombie_zz
17:07:17  * mikolalysenkoquit (Ping timeout: 240 seconds)
17:09:10  * mikolalysenkojoined
17:19:22  * CoverSlidejoined
17:31:30  * slaskisquit (Ping timeout: 276 seconds)
17:37:25  * CoverSlidequit (Ping timeout: 260 seconds)
17:53:10  * jden__quit (Ping timeout: 260 seconds)
17:55:19  * shamajoined
17:56:47  * gwenbelljoined
17:58:33  * mikealquit (Quit: Leaving.)
17:59:18  * mikealjoined
18:04:50  * duncanbeeversquit (Read error: Operation timed out)
18:10:25  * tphummeljoined
18:15:44  <jjjjohnnny>how to make a tube amplifier
18:17:10  <jjjjohnnny>nope
18:20:43  * dominictarrquit (Quit: dominictarr)
18:26:50  * CoverSlidejoined
18:32:46  * mikolalysenkoquit (Ping timeout: 240 seconds)
18:34:58  * shamaquit (Remote host closed the connection)
18:36:05  * CryptoQuickjoined
18:41:37  * defunctzombie_zzchanged nick to defunctzombie
18:42:59  * mikolalysenkojoined
18:44:07  * dguttmanjoined
18:48:44  <rowbit>/!\ ATTENTION: (default-local) [email protected] successfully signed up for developer browserling plan ($20). Cash money! /!\
18:48:44  <rowbit>/!\ ATTENTION: (default-local) paid account successfully upgraded /!\
18:48:48  * slaskisjoined
18:49:54  * AvianFluquit (Remote host closed the connection)
18:51:05  * AvianFlujoined
18:51:27  * dguttmanquit (Quit: dguttman)
19:00:01  * jden_joined
19:00:24  * jden_changed nick to Guest56812
19:03:06  * nk109joined
19:04:55  * Guest56812quit (Ping timeout: 260 seconds)
19:07:39  * gwenbellquit (Remote host closed the connection)
19:11:34  * tilgovijoined
19:15:16  * slaskisquit (Remote host closed the connection)
19:16:09  * lyaunzbequit (Remote host closed the connection)
19:18:26  <defunctzombie>isaacs: cache.add(what) ... is there a cache.read(what) ?
19:18:36  <defunctzombie>the only cache.read I see doesn't take the same form of 'what'
19:19:32  <defunctzombie>I want a way to ask the cache if it already has something with the same format that cache.add wants when adding something
19:20:13  <isaacs>defunctzombie: don't know offhand.
19:20:36  <Raynos>rvagg: oh hi
19:21:06  * slaskisjoined
19:21:07  <defunctzombie>isaacs: I have some new install functionality (very crude) but one thing that is lacking is a good way to check if something is already installed or available in the cache that is consistent with how I add items to the cache
19:22:02  <isaacs>defunctzombie: i'd have to read the code to tell you.
19:22:06  <isaacs>defunctzombie: you can read the code :)
19:22:18  <defunctzombie>isaacs: I am not sure that is a true statement :p
19:22:48  * dguttmanjoined
19:24:43  <isaacs>defunctzombie: shake free of your jslint brain!
19:24:57  <isaacs>seriously, cache.js isn't so bad.
19:24:59  <isaacs>to read.
19:25:04  <isaacs>it's just functions that call other functions.
19:25:07  <defunctzombie>I don't think it is my jslint brain is what is holding me back
19:25:16  <defunctzombie>isaacs: ... really...
19:25:16  <isaacs>and all roads lead to addPlacedTarball
19:25:27  <isaacs>it's too long, i mean, it's like a whole program in one file.
19:25:30  <isaacs>it should be its own module.
19:25:47  <isaacs>and the actual *approach* leaves much to be desired (as we've discussed)
19:25:55  <isaacs>but the implementation of that approach is not terrible.
19:27:08  <defunctzombie>cache.add works quite well, I like that part
19:27:14  <defunctzombie>I just need to make the counterpart
19:27:21  <defunctzombie>which doesn't seem to exist
19:27:55  <defunctzombie>cache.add does what I want, fetches whatever and makes a target I can call cache.unpack on
19:28:27  <defunctzombie>I just need a way to get that target without actually doing the add if it already exists :/
19:28:54  * slaskisquit (Remote host closed the connection)
19:32:58  * dguttmanquit (Read error: Connection reset by peer)
19:36:13  * dguttmanjoined
19:40:28  * dguttmanquit (Client Quit)
19:55:07  * defunctzombiechanged nick to defunctzombie_zz
20:03:34  * duncanbeeversjoined
20:09:20  * lyaunzbejoined
20:29:17  * AvianFluquit (Remote host closed the connection)
21:02:39  * defunctzombie_zzchanged nick to defunctzombie
21:13:22  * evboguejoined
21:25:15  * lyaunzbequit (Remote host closed the connection)
21:34:30  * lyaunzbejoined
22:00:24  * shamajoined
22:10:39  * dguttmanjoined
22:14:20  * jden_joined
22:14:44  * jden_changed nick to Guest57980
22:36:09  <defunctzombie>seriously.. why are people talking about this opera webkit thing still
22:36:24  <defunctzombie>I woke up this morning, saw the news, and MOVED THE FUCK ON
22:42:09  * lyaunzbequit (Remote host closed the connection)
22:58:30  * lyaunzbejoined
23:06:27  <Raynos>because its sads
23:06:28  <Raynos>many sads
23:06:32  <Raynos>because opera bros are cool
23:06:35  <Raynos>and they threw in the towel
23:10:15  <CoverSlide>they are cool, and they will contribute to webkit upstream
23:10:59  <CoverSlide>it's better for them to dedicate their efforts to the more widely-used platform instead of wasting time on theirs which doesn't get quite as much use
23:13:14  <CoverSlide>so when they do cool proposals such as webrtc, it gets implemented on 50x more browsers than it would have been if they had done it one their old one. no biggie
23:15:44  * AvianFlujoined
23:19:37  * blobaumjoined
23:25:12  <Raynos>anyone have a TTL cache thing?
23:29:04  * dguttman_joined
23:29:25  * shamaquit (Remote host closed the connection)
23:30:20  * evboguequit (Read error: Connection reset by peer)
23:30:59  * dguttmanquit (Ping timeout: 260 seconds)
23:30:59  * dguttman_changed nick to dguttman
23:47:30  * defunctzombiechanged nick to defunctzombie_zz