00:11:43  * tarrudaquit (Read error: Connection reset by peer)
00:31:28  * jerrysvquit (Remote host closed the connection)
00:32:01  * jerrysvjoined
00:36:29  * jerrysvquit (Ping timeout: 245 seconds)
00:48:59  * anton__part ("pepperland")
00:49:36  * no9joined
01:32:09  * DTrejojoined
01:32:18  * jerrysvjoined
01:34:15  * DTrejoquit (Remote host closed the connection)
01:37:17  * jerrysvquit (Ping timeout: 246 seconds)
02:03:10  * mhernandez1quit (Remote host closed the connection)
02:33:03  * jerrysvjoined
02:38:04  * jerrysvquit (Ping timeout: 255 seconds)
03:14:48  * dguttmanquit (Quit: dguttman)
03:17:37  * mhernandez1joined
03:33:52  * jerrysvjoined
03:38:49  * jerrysvquit (Ping timeout: 255 seconds)
04:16:20  * mhernandez1quit (Remote host closed the connection)
04:34:36  * jerrysvjoined
04:39:38  * jerrysvquit (Ping timeout: 244 seconds)
04:49:18  * binocarlosjoined
05:00:42  * dguttmanjoined
05:35:44  * jerrysvjoined
05:36:09  * dguttmanquit (Quit: dguttman)
05:40:22  * jerrysvquit (Ping timeout: 240 seconds)
05:56:06  * dguttmanjoined
06:03:12  * binocarlosquit (Ping timeout: 245 seconds)
06:03:29  * eugeneware_quit (Ping timeout: 252 seconds)
06:04:56  * erikgquit (Ping timeout: 252 seconds)
06:05:43  * erikgjoined
06:07:18  * eugeneware_joined
06:07:57  * stagasjoined
06:32:07  * dguttmanquit (Quit: dguttman)
06:36:27  * jerrysvjoined
06:41:05  * jerrysvquit (Ping timeout: 246 seconds)
07:37:14  * jerrysvjoined
07:42:14  * jerrysvquit (Ping timeout: 264 seconds)
08:22:58  * stagasquit (Ping timeout: 250 seconds)
08:27:52  * bonswouarjoined
08:37:43  * jerrysvjoined
08:42:34  * jerrysvquit (Ping timeout: 255 seconds)
09:15:06  * joakinojoined
09:38:24  * jerrysvjoined
09:42:53  * jerrysvquit (Ping timeout: 245 seconds)
09:48:40  <jcrugzz>anyone alive around these parts?
09:49:44  <jcrugzz>have the weirdest bug im battling
09:50:10  * tarrudajoined
09:58:36  <kenansulayman>jcrugzz which?
10:01:27  <jcrugzz>kenansulayman: for whatever reason my json value is consistently being encoded as [object Object] and then throws an encoding error when trying to be read because of this
10:01:40  <jcrugzz>trying to figure out why my batches are being inserted with bad values
10:01:50  <kenansulayman>LevelDB _does'nt_ store Objects
10:02:03  <kenansulayman>You're required to JSON.stringify your objects
10:02:26  <kenansulayman>and subsequently JSON.parse when getting a value from the database
10:03:02  <kenansulayman>Are you specificying "json" as encoding when putting your document?
10:03:22  <jcrugzz>kenansulayman: im explicitly specifying the json valueEncoding https://github.com/rvagg/node-levelup#json-data
10:03:59  <kenansulayman>Okay, then indeed it should work. May I see the code? PM if private code
10:04:24  <jcrugzz>on instantiation of the database as well as with the batches themselves. trying to do some logging in the batch function currently. let me see if i can get some output
10:05:46  <jcrugzz>kenansulayman: the core pipeline im using is the following https://github.com/jcrugzz/level-batch-stream. minus concurrent-writable atm
10:05:53  <jcrugzz>my example
10:07:45  <kenansulayman>I'm afraid you need to specify the value encoding per key in a batch
10:08:40  <jcrugzz>kenansulayman: it inherits the valueEncoding from the database, and the behavior didnt change when i explicitly specified it in each key within that transform stream
10:09:06  <jcrugzz>ive been looking at this for 2 hrs. its seemingly straightforward but idk wtf is happening lol
10:09:23  <kenansulayman>hah, I can imagine. Looking into it
10:10:37  * tarrudaquit (Ping timeout: 264 seconds)
10:10:52  <jcrugzz>kenansulayman: i really appreciate it
10:26:48  <kenansulayman>jcrugzz fun fact: your code doesn't work with 1.2.0 that I am testing with. It's not able to emit objects and only accepts strings :-E
10:26:54  <kenansulayman>What node are you using?
10:28:17  <jcrugzz>kenansulayman: hmm got it there is a bug in my documentation. it should be through.obj instead of through. one second and ill give you a full gist of what im doing
10:28:39  <kenansulayman>Okay!
10:33:34  <jcrugzz>kenansulayman: https://gist.github.com/jcrugzz/2943d9fe088d923d102b
10:33:56  <kenansulayman>Thanks
10:34:20  <jcrugzz>because i fail, the first time it doesnt actually show data
10:34:24  <jcrugzz>but the second run does
10:34:35  <jcrugzz>and those damn values are [object Object]
10:34:39  <jcrugzz>maddening!
10:39:08  * jerrysvjoined
10:42:59  <kenansulayman>jcrugzz that batch interface is pretty fucked up. it doesn't tolerate any encoding
10:43:36  <kenansulayman>That is, it enters my custom valueEncoding method on reading the key, which consequently fails (since it didn't enter the encode that stringified it)
10:43:56  * jerrysvquit (Ping timeout: 246 seconds)
10:44:02  <kenansulayman>That said, it doesn
10:44:26  <kenansulayman>'t accept strings, so I assume it's a bug in another module you're using
10:46:25  <kenansulayman>"enc" in the callback of through.obj is "utf8". is that what you expect?
10:49:32  <jcrugzz>kenansulayman: that should be fine
10:49:46  <jcrugzz>because ive confirmed objects get passed down all the way to my level-batch-stream module
10:50:11  <jcrugzz>something just goes awry when it gets batched internally there
10:50:24  <jcrugzz>which is concerning, cause it feels all terribly wrong
10:52:30  * gyaresuquit (Ping timeout: 276 seconds)
10:59:30  <kenansulayman>jcrugzz what is that this.db you're using?
11:00:23  <kenansulayman>it only has the .batch method..
11:00:51  <kenansulayman>At least I'm unable to self.db.createReadStream() from inside this.batch
11:02:27  <jcrugzz>kenansulayman, the passed in instance of levelup.
11:05:03  <kenansulayman>That is strange. In the top scope of the script, there's a db.createReadStream() but not in the this.batch method of your module
11:05:07  <kenansulayman>investigating..
11:06:25  <jcrugzz>hmmm
11:08:24  <kenansulayman>jcrugzz http://data.sly.mn/image/0t2p2G3k0m1C
11:10:58  <jcrugzz>kenansulyman: unfortunately i think its unrelated but that maybe should be handled in deferred leveldown
11:10:59  <jcrugzz>https://github.com/Level/deferred-leveldown/blob/master/deferred-leveldown.js
11:11:10  <jcrugzz>that method just doesnt get proxied
11:11:21  <jcrugzz>probably because that can get complex
11:11:51  <jcrugzz>but it could have something to do with the deferred logic
11:11:55  <kenansulayman>good lord
11:12:13  <jcrugzz>like not going through the encoding stuff if its called while opening?
11:12:21  <jcrugzz>hmm lets try waiting until it calls open
11:12:26  <jcrugzz>MAYBE that does something
11:12:31  <kenansulayman>yes..
11:12:35  <kenansulayman>try it
11:17:39  <jcrugzz>man
11:17:41  <jcrugzz>this is some insane shit
11:17:53  <jcrugzz>my console.logs dont execute
11:17:58  <jcrugzz>thats crazy to me
11:18:06  <jcrugzz>in the batch function in levelup
11:19:57  <jcrugzz>kenansulayman: i may have to give up on this one tonight
11:19:59  <jcrugzz>but wtf
11:20:08  <jcrugzz>this is literal insanity to me
11:20:29  <kenansulayman>haha for me as well. but I have to do actual work today, this is sparta :-E
11:22:51  <jcrugzz>yea its like 3:30 am here
11:23:03  <jcrugzz>writing tests for something gets me here ....
11:23:08  <jcrugzz>i dont even
11:39:53  * jerrysvjoined
11:44:32  * jerrysvquit (Ping timeout: 246 seconds)
12:41:06  * jerrysvjoined
12:45:50  * jerrysvquit (Ping timeout: 264 seconds)
13:41:44  * jerrysvjoined
13:46:00  * blessYahujoined
13:46:05  * tarrudajoined
13:46:19  * jerrysvquit (Ping timeout: 255 seconds)
13:58:37  * mhernandez1joined
14:42:12  * jerrysvjoined
14:44:33  * blessYahuquit (Remote host closed the connection)
14:45:58  * rudjoined
14:45:59  * rudquit (Changing host)
14:45:59  * rudjoined
14:47:11  * jerrysvquit (Ping timeout: 246 seconds)
14:47:20  * blessYahujoined
14:48:12  * neonstalwartjoined
14:51:17  * blessYahuquit (Remote host closed the connection)
14:55:22  * blessYahujoined
15:15:59  * dguttmanjoined
15:42:57  * jerrysvjoined
15:47:45  * jerrysvquit (Ping timeout: 246 seconds)
15:52:52  * tarrudaquit (Quit: WeeChat 1.0.1)
16:05:46  * jerrysvjoined
16:13:25  * bonswouarquit (Remote host closed the connection)
16:19:28  * dguttmanquit (Read error: No route to host)
16:22:59  * dguttmanjoined
16:44:33  * rudquit (Quit: rud)
17:36:12  * tolgayjoined
17:37:38  <tolgay>hi
17:37:41  <tolgay>can i ask a question ?
17:37:51  <tolgay>about node.js
17:37:55  <tolgay>and leveldb
17:42:14  * DTrejojoined
17:46:13  <Aria>Definitely.
17:56:22  <tolgay>can you read it from this link ? http://stackoverflow.com/questions/28461092/data-stream-from-database-in-node-js
18:50:14  <gildean>tolgay: why do you need two different node processes?
18:57:39  <tolgay>I will explain my system
18:57:42  <tolgay>are you there ?
19:01:33  * dcbjoined
19:27:53  * dcbquit (Ping timeout: 245 seconds)
19:29:08  * jerrysv_joined
19:30:07  * jerrysvquit (Read error: Connection reset by peer)
19:30:10  * jerrysv__joined
19:33:46  * jerrysv_quit (Ping timeout: 250 seconds)
19:34:36  * jerrysv__changed nick to jerrysv
19:35:24  <gildean>tolgay: yes, but not all the time
19:35:55  <tolgay>could you read this ? http://stackoverflow.com/questions/28462731/node-js-and-message-delivery-acknowledgement
19:47:14  * DTrejoquit (Remote host closed the connection)
19:58:45  <gildean>tolgay: if you're tracking the messages with something unique (like an id), you could just use an object to save a timeout for each message that gets cleared when the other user sends the ack
19:59:31  <tolgay>yes i am using unique id. can you explain more ?
20:00:12  <gildean>tolgay: let me show you a quick example
20:00:25  <tolgay>thank you so much,i am waiting
20:04:06  <neonstalwart>tolgay: i have something similar which queues messages for clients and if they haven't cleared their queue for more than 15 seconds then i stop assume they've disconnected and stop keeping messages for them.
20:04:41  <tolgay>awesome! can you explain me your system ?
20:05:44  <neonstalwart>the way it works… timestamp the queue with the last time it was read. keep a list of the queues sorted by this timestamp and then periodically (about every second) walk that list and remove all the queues which have a timestamp older than 15 seconds
20:07:22  <tolgay>are you using node.js ?
20:07:25  <neonstalwart>yes
20:07:30  <tolgay>and leveldb ?
20:07:39  <neonstalwart>yes, but leveldb is irrelevant here
20:07:51  <neonstalwart>the list of queues is in memory
20:08:09  <tolgay>where are you storing the messages ?
20:08:21  <tolgay>if i reboot the machine ?
20:08:38  <neonstalwart>that's not necessarily relevant - you can store them in memory or in a db. it all depends on your needs
20:09:07  <neonstalwart>if you want to keep the list of queues in leveldb you can do that too
20:09:13  <gildean>tolgay: my first idea was basically just this: http://jsfiddle.net/8fny0c66/
20:09:20  <tolgay>could you show me little bit code ? for creating queue
20:09:30  <gildean>the example uses dom-objects but the same idea would work with anything
20:09:42  <neonstalwart>you don't want a timer per connection
20:09:55  <neonstalwart>that is inefficient for a large number of connections
20:10:23  <neonstalwart>this is why you want the timestamp and a single timer
20:11:18  <tolgay>hmmm i am trying it now
20:11:27  <gildean>yeah, that would work much better when the number of messages grows
20:11:37  <gildean>with only a few messages it wouldn't really matter
20:12:09  * joakinoquit
20:12:10  <tolgay>my system will use by 2k concurrent users
20:12:14  <tolgay>what do you suggest ?
20:13:24  * dguttman_joined
20:14:13  * dguttmanquit (Ping timeout: 255 seconds)
20:14:13  * dguttman_changed nick to dguttman
20:17:23  <neonstalwart>a timestamp and a single timer. you may want to use the timestamp as part of your key so that you can query by timestamp and just get the messages which have expired
20:17:57  <tolgay>i am trying it now
20:19:28  * ncthom91joined
20:35:49  * mhernandez1quit (Remote host closed the connection)
20:46:02  * mhernandez1joined
21:01:47  * ncthom91quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
21:08:14  * mhernandez1quit (Remote host closed the connection)
21:09:32  * mhernandez1joined
21:26:07  * mhernandez1quit (Ping timeout: 245 seconds)
21:29:23  * mhernandez1joined
21:38:23  * mhernandez1quit (Read error: Connection reset by peer)
21:46:46  * ncthom91joined
21:52:40  * mitzipquit (Quit: good bye)
21:52:58  * mitzipjoined
22:11:37  * ncthom91quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
22:58:07  * tolgayquit (Quit: Page closed)
23:17:22  * neonstalwartquit (Quit: Leaving.)
23:49:49  * ncthom91joined
23:50:21  * ncthom91quit (Client Quit)