MBR IT/.NET 247 : local XML file or trip to SQL DB? on ASPFriends.com 'aspngarchitecture' list

local XML file or trip to SQL DB?

Messages

Related Types

This message was discovered on ASPFriends.com 'aspngarchitecture' list.

Francesco Sanfilippo

Which is faster: reading and writing to and from a tiny XML file on the web
server, or reading and writing to and from the SQL DB on the same box as the
web server?

The purpose is to simulate IIS Logging, but with some custom items I need.
I don't wantto make a trip to the DB every click of every user. I think it
would be better to accumulate the strings somewhere faster, and every 15
minutes or so, dump it to SQL.

Francesco

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp.

Reply to this message...

Tim Royal

If I were you, and I wanted to track that kind of activity, I'd look into
having an application scope class with a static method (and static
StringBuilder object), and just append a list of data to that on each
request. Maybe run a process that dumps it to a database every X minutes.
THe only thing I can think of that may be an issue is the concurrency issue,
but then you have that with file read/writes anyway (something you'd be
assured of not happening with directly inputting your info into SQL).

In reality, you could do start off by writing the process to hit SQL Server
and see if it can handle the load you anticipate. If not, you try the other
approaches next (since you'll need the SQL Server anyway to do the batch
uploads, you won't lose much, if any, work in doing the Direct to SQL Server
approach).

Tim

->-----Original Message-----
->From: Francesco Sanfilippo [mailto:Click here to reveal e-mail address]
->Sent: Thursday, June 06, 2002 3:45 PM
->To: aspngarchitecture
->Subject: [aspngarchitecture] local XML file or trip to SQL DB?
->
->
->Which is faster: reading and writing to and from a tiny XML
->file on the web
->server, or reading and writing to and from the SQL DB on the
->same box as the
->web server?
->
->The purpose is to simulate IIS Logging, but with some custom
->items I need.
->I don't wantto make a trip to the DB every click of every
->user. I think it
->would be better to accumulate the strings somewhere faster,
->and every 15
->minutes or so, dump it to SQL.
->
->Francesco
->
->_________________________________________________________________
->Get your FREE download of MSN Explorer at
->http://explorer.msn.com/intl.asp.
->
->
->| [aspngarchitecture] member Click here to reveal e-mail address = YOUR ID
->| http://www.asplists.com/asplists/aspngarchitecture.asp = JOIN/QUIT
->| http://www.asplists.com/search = SEARCH Archives
->
->
->

Reply to this message...

Dan Wahlin

You can find a simple comparison test at the following URL:

http://www.xmlforasp.net/codeSection.aspx?csID=5

Although not scientific, you can get a general idea about how different
XML parsing techniques compare with hitting a db. The source code is
available so you can tweak things as needed.

HTH,
Dan Wahlin

Wahlin Consulting LLC
Microsoft MVP - ASP.NET
http://www.XMLforASP.Net: #1 ASP.NET XML Resource
XML for ASP.NET Developers by Dan Wahlin in bookstores everywhere!

-----Original Message-----
From: Francesco Sanfilippo [mailto:Click here to reveal e-mail address]
Sent: Thursday, June 06, 2002 3:45 PM
To: aspngarchitecture
Subject: [aspngarchitecture] local XML file or trip to SQL DB?

Which is faster: reading and writing to and from a tiny XML file on the
web
server, or reading and writing to and from the SQL DB on the same box as
the
web server?

The purpose is to simulate IIS Logging, but with some custom items I
need.
I don't wantto make a trip to the DB every click of every user. I think
it
would be better to accumulate the strings somewhere faster, and every 15

minutes or so, dump it to SQL.

Francesco

_________________________________________________________________
Get your FREE download of MSN Explorer at
http://explorer.msn.com/intl.asp.

| [aspngarchitecture] member Click here to reveal e-mail address = YOUR ID
| http://www.asplists.com/asplists/aspngarchitecture.asp = JOIN/QUIT
| http://www.asplists.com/search = SEARCH Archives

Reply to this message...

Russ McClelland

SSB3YXMgdGhpbmtpbmcgdGhlIHNhbWUgdGhpbmcgb25seSBoYXZpbmcgdGhlIHRyYWNlIGV2ZW50
cyBhdXRvbWF0aWNhbGx5DQpsb2dnZWQgd2hlbiB0aGUgc2Vzc2lvbiBjbG9zZWQvdGVybWluYXRl
ZC4gIEF2b2lkcyB0aGUgbmVlZCBmb3IgY29kZSB0bw0KY2hlY2sgdGltZSBpbnRlcnZhbHMgb3Ig
ZXZlbnQgdGhyZXNoaG9sZHMgKGV0Yy4pIGZvciBzaW1wbGljaXRpZXMgc2FrZS4NCkkgZG91YnQg
eW91IHdvdWxkIGxvc2UgdG9vIG1hbnkgc2Vzc2lvbnMgaW4gYSB3YXkgdGhhdCB3b3VsZG4ndCBn
ZXQNCmNsZWFuZWQgdXAgZmlyc3QuDQoNCgktLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLSANCglG
cm9tOiBUaW0gUm95YWwgDQoJU2VudDogVGh1IDYvNi8yMDAyIDExOjAzIFBNIA0KCVRvOiBhc3Bu
Z2FyY2hpdGVjdHVyZSANCglDYzogDQoJU3ViamVjdDogW2FzcG5nYXJjaGl0ZWN0dXJlXSBSRTog
bG9jYWwgWE1MIGZpbGUgb3IgdHJpcCB0byBTUUwNCkRCPw0KCQ0KCQ0KDQoJSWYgSSB3ZXJlIHlv
dSwgYW5kIEkgd2FudGVkIHRvIHRyYWNrIHRoYXQga2luZCBvZiBhY3Rpdml0eSwgSSdkDQpsb29r
IGludG8NCgloYXZpbmcgYW4gYXBwbGljYXRpb24gc2NvcGUgY2xhc3Mgd2l0aCBhIHN0YXRpYyBt
ZXRob2QgKGFuZA0Kc3RhdGljDQoJU3RyaW5nQnVpbGRlciBvYmplY3QpLCBhbmQganVzdCBhcHBl
bmQgYSBsaXN0IG9mIGRhdGEgdG8gdGhhdCBvbg0KZWFjaA0KCXJlcXVlc3QuIE1heWJlIHJ1biBh
IHByb2Nlc3MgdGhhdCBkdW1wcyBpdCB0byBhIGRhdGFiYXNlIGV2ZXJ5IFgNCm1pbnV0ZXMuDQoJ
VEhlIG9ubHkgdGhpbmcgSSBjYW4gdGhpbmsgb2YgdGhhdCBtYXkgYmUgYW4gaXNzdWUgaXMgdGhl
DQpjb25jdXJyZW5jeSBpc3N1ZSwNCglidXQgdGhlbiB5b3UgaGF2ZSB0aGF0IHdpdGggZmlsZSBy
ZWFkL3dyaXRlcyBhbnl3YXkgKHNvbWV0aGluZw0KeW91J2QgYmUNCglhc3N1cmVkIG9mIG5vdCBo
YXBwZW5pbmcgd2l0aCBkaXJlY3RseSBpbnB1dHRpbmcgeW91ciBpbmZvIGludG8NClNRTCkuDQoJ
DQoJSW4gcmVhbGl0eSwgeW91IGNvdWxkIGRvIHN0YXJ0IG9mZiBieSB3cml0aW5nIHRoZSBwcm9j
ZXNzIHRvIGhpdA0KU1FMIFNlcnZlcg0KCWFuZCBzZWUgaWYgaXQgY2FuIGhhbmRsZSB0aGUgbG9h
ZCB5b3UgYW50aWNpcGF0ZS4gSWYgbm90LCB5b3UNCnRyeSB0aGUgb3RoZXINCglhcHByb2FjaGVz
IG5leHQgKHNpbmNlIHlvdSdsbCBuZWVkIHRoZSBTUUwgU2VydmVyIGFueXdheSB0byBkbw0KdGhl
IGJhdGNoDQoJdXBsb2FkcywgeW91IHdvbid0IGxvc2UgbXVjaCwgaWYgYW55LCB3b3JrIGluIGRv
aW5nIHRoZSBEaXJlY3QNCnRvIFNRTCBTZXJ2ZXINCglhcHByb2FjaCkuDQoJDQoJVGltDQoJDQoJ
DQoJDQoJLT4tLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KCS0+RnJvbTogRnJhbmNlc2NvIFNh
bmZpbGlwcG8gWyBtYWlsdG86cGhvdG9yZWFsbUBob3RtYWlsLmNvbV0NCgktPlNlbnQ6IFRodXJz
ZGF5LCBKdW5lIDA2LCAyMDAyIDM6NDUgUE0NCgktPlRvOiBhc3BuZ2FyY2hpdGVjdHVyZQ0KCS0+
U3ViamVjdDogW2FzcG5nYXJjaGl0ZWN0dXJlXSBsb2NhbCBYTUwgZmlsZSBvciB0cmlwIHRvIFNR
TCBEQj8NCgktPg0KCS0+DQoJLT5XaGljaCBpcyBmYXN0ZXI6ICByZWFkaW5nIGFuZCB3cml0aW5n
IHRvIGFuZCBmcm9tIGEgdGlueSBYTUwNCgktPmZpbGUgb24gdGhlIHdlYg0KCS0+c2VydmVyLCBv
ciByZWFkaW5nIGFuZCB3cml0aW5nIHRvIGFuZCBmcm9tIHRoZSBTUUwgREIgb24gdGhlDQoJLT5z
YW1lIGJveCBhcyB0aGUNCgktPndlYiBzZXJ2ZXI/DQoJLT4NCgktPlRoZSBwdXJwb3NlIGlzIHRv
IHNpbXVsYXRlIElJUyBMb2dnaW5nLCBidXQgd2l0aCBzb21lIGN1c3RvbQ0KCS0+aXRlbXMgSSBu
ZWVkLg0KCS0+SSBkb24ndCB3YW50dG8gbWFrZSBhIHRyaXAgdG8gdGhlIERCIGV2ZXJ5IGNsaWNr
IG9mIGV2ZXJ5DQoJLT51c2VyLiAgSSB0aGluayBpdA0KCS0+d291bGQgYmUgYmV0dGVyIHRvIGFj
Y3VtdWxhdGUgdGhlIHN0cmluZ3Mgc29tZXdoZXJlIGZhc3RlciwNCgktPmFuZCBldmVyeSAxNQ0K
CS0+bWludXRlcyBvciBzbywgZHVtcCBpdCB0byBTUUwuDQoJLT4NCgktPkZyYW5jZXNjbw0KCS0+
DQoJDQotPl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fDQoJLT5HZXQgeW91ciBGUkVFIGRvd25sb2FkIG9mIE1TTiBFeHBsb3Jl
ciBhdA0KCS0+IGh0dHA6Ly9leHBsb3Jlci5tc24uY29tL2ludGwuYXNwLg0KCS0+DQoJLT4NCgkt
PnwgW2FzcG5nYXJjaGl0ZWN0dXJlXSBtZW1iZXIgdGltckBodWxhYmVlLmNvbSA9IFlPVVIgSUQN
CgktPnwgaHR0cDovL3d3dy5hc3BsaXN0cy5jb20vYXNwbGlzdHMvYXNwbmdhcmNoaXRlY3R1cmUu
YXNwID0NCkpPSU4vUVVJVA0KCS0+fCBodHRwOi8vd3d3LmFzcGxpc3RzLmNvbS9zZWFyY2ggPSBT
RUFSQ0ggQXJjaGl2ZXMNCgktPg0KCS0+DQoJLT4NCgkNCgkNCgl8IFthc3BuZ2FyY2hpdGVjdHVy
ZV0gbWVtYmVyIHJ1c3MubWNjbGVsbGFuZEBzbWFydG9ianguY29tID0NCllPVVIgSUQNCgl8IGh0
dHA6Ly93d3cuYXNwbGlzdHMuY29tL2FzcGxpc3RzL2FzcG5nYXJjaGl0ZWN0dXJlLmFzcCA9DQpK
T0lOL1FVSVQNCgl8IGh0dHA6Ly93d3cuYXNwbGlzdHMuY29tL3NlYXJjaCA9IFNFQVJDSCBBcmNo
aXZlcw0KCQ0KCQ0KDQo

Reply to this message...

Mark Feinholz

If you were to manage this in memory, with an object scoped to the
application level, and wanted to make sure that that object got
persisted to disk (either to a db or a file) every 5 minutes or so, what
techniques are available for making sure that every 5 minutes some
thread wakes up and dumps the application object to disk?

In other words, how do you set up a thread that wakes up every 5 minutes
and has access to the Application object?

Mark.

-----Original Message-----
From: Tim Royal [mailto:Click here to reveal e-mail address]
Sent: Thursday, June 06, 2002 9:03 PM
To: aspngarchitecture
Subject: [aspngarchitecture] RE: local XML file or trip to SQL DB?

If I were you, and I wanted to track that kind of activity, I'd look
into
having an application scope class with a static method (and static
StringBuilder object), and just append a list of data to that on each
request. Maybe run a process that dumps it to a database every X
minutes.
THe only thing I can think of that may be an issue is the concurrency
issue,
but then you have that with file read/writes anyway (something you'd be
assured of not happening with directly inputting your info into SQL).

In reality, you could do start off by writing the process to hit SQL
Server
and see if it can handle the load you anticipate. If not, you try the
other
approaches next (since you'll need the SQL Server anyway to do the batch
uploads, you won't lose much, if any, work in doing the Direct to SQL
Server
approach).

Tim

->-----Original Message-----
->From: Francesco Sanfilippo [mailto:Click here to reveal e-mail address]
->Sent: Thursday, June 06, 2002 3:45 PM
->To: aspngarchitecture
->Subject: [aspngarchitecture] local XML file or trip to SQL DB?
->
->
->Which is faster: reading and writing to and from a tiny XML
->file on the web
->server, or reading and writing to and from the SQL DB on the
->same box as the
->web server?
->
->The purpose is to simulate IIS Logging, but with some custom
->items I need.
->I don't wantto make a trip to the DB every click of every
->user. I think it
->would be better to accumulate the strings somewhere faster,
->and every 15
->minutes or so, dump it to SQL.
->
->Francesco
->
->_________________________________________________________________
->Get your FREE download of MSN Explorer at
->http://explorer.msn.com/intl.asp.
->
->
->| [aspngarchitecture] member Click here to reveal e-mail address = YOUR ID
->| http://www.asplists.com/asplists/aspngarchitecture.asp = JOIN/QUIT
->| http://www.asplists.com/search = SEARCH Archives
->
->
->

| [aspngarchitecture] member Click here to reveal e-mail address = YOUR ID
| http://www.asplists.com/asplists/aspngarchitecture.asp = JOIN/QUIT
| http://www.asplists.com/search = SEARCH Archives

Reply to this message...

mail

Not everything that glitters is gold!!!!!!!!

I saw a very interesting conversation between Paul Flessner (Vice
Presidente de SQL Server y Middleware), Joe Long (Group Manager de COM+)
y Robert Hess (Product Manager para Microsoft y presentador del .NET
Show).
Here they talk about this issue and I agree with them

http://msdn.microsoft.com/theshow/Viewer/player.asp?
folder=Episode003&file=56k.asx

-----Original Message-----
From: "Francesco Sanfilippo" <Click here to reveal e-mail address>
To: "aspngarchitecture" <Click here to reveal e-mail address>
Date: Thu, 06 Jun 2002 15:45:19 -0700
Subject: [aspngarchitecture] local XML file or trip to SQL DB?

[Original message clipped]

Reply to this message...

Ben Hyrman

Have you considered firing off events in response to user actions you want
to record? These could be fired asyncronously (no noticable overhead to the
user) and then recorded to the storage structure of your choice by a
listener. Basically, record to a queue, read from a queue. This is how many
of the app servers I've worked with have handled your problem.

Ben

----- Original Message -----
From: "Francesco Sanfilippo" <Click here to reveal e-mail address>
To: "aspngarchitecture" <Click here to reveal e-mail address>
Sent: Thursday, June 06, 2002 3:45 PM
Subject: [aspngarchitecture] local XML file or trip to SQL DB?

> Which is faster: reading and writing to and from a tiny XML file on the
web
> server, or reading and writing to and from the SQL DB on the same box as
the
[Original message clipped]

Reply to this message...

System.Text.StringBuilder

MBR BootFX
Best-of-breed application framework for .NET projects, developed by Matthew Baxter-Reynolds and MBR IT

	Copyright © Matthew Baxter-Reynolds 2001-2008. '.NET 247 Software Development Services' is a trading style of MBR IT Solutions Ltd.
	Contact Us - Terms of Use - Privacy Policy - www.dotnet247.com