Jump to content

Welcome to Geeks to Go - Register now for FREE

Geeks To Go is a helpful hub, where thousands of volunteer geeks quickly serve friendly answers and support. Check out the forums and get free advice from the experts. Register now to gain access to all of our features, it's FREE and only takes one minute. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more.

Create Account How it Works
Photo

accessing asci text files


  • Please log in to reply

#1
rambleon

rambleon

    New Member

  • Member
  • Pip
  • 3 posts
Hi,
I have an asp.net intranet app. that allows employees to view their payslips for the last 5 years. The payslip info. is in a monthly flat text file. I have a database that contains the employees id number, the file name and the line number within the file where the employees payslip starts. The company has about 5000 employees, each file contains about 300,000 lines, for each payslip retrieved I have to open the file, read to the line of the employees payslip, present the payslip and close the file. In testing the performance is reasonable. My question is how will using this method in an intranet environment where many users are reading the same text file at the same time impact on performance and if anyone can suggest any alternative methods or ways to improve performance.

Thanks

Rambleon
  • 0

Advertisements


#2
TaNkZ101

TaNkZ101

    Member

  • Member
  • PipPipPip
  • 327 posts
hi there,
i don't know the answer to your question, but i think you should post this in a hardware forum (perhaps networking?) and you might get more help there
  • 0

#3
ScHwErV

ScHwErV

    Member 5k

  • Retired Staff
  • 21,285 posts
  • MVP
Since its ASP and database related, it belongs here.

Flatfiles are always going to be a slower idea than running an sql database. If your database was sql based, you could have it run queries and only display the necessary information versus all 300,000 lines.

Any time you run a single file that hundreds of people will access at the same time, you are asking for trouble.

ScHwErV :whistling:
  • 0

#4
rambleon

rambleon

    New Member

  • Topic Starter
  • Member
  • Pip
  • 3 posts
The actual location of the text within the file is in an sql database. what I was wondering if it's possible to directly arrive to the 100,000 line by specifying the offset within the file (in bytes) or do i have to read 99,999 lines first ?
The other thing is does each user have exclusive access to the file or can several users open and read the file at the same time ?
  • 0

#5
TDXkev

TDXkev

    Member

  • Member
  • PipPip
  • 24 posts
If possible, you should lose the text file and put everything in your database. Otherwise, you're performance is not going to be that great if a lot of people are accessing the file at once, especially if you are reading in each line until you get to the one you want.

It might take time to input it into a database, but it will be worth it in the long run as it is much more maintainable than a flat file.

In summary:

1. The file I/O will hurt your performance.
2. The database queries would be much faster.
3. The database structure is a better idea for maintainability in the long run.
4. The database structure will be much easier to manipulate.

Good Luck! :whistling:
  • 0

#6
ScHwErV

ScHwErV

    Member 5k

  • Retired Staff
  • 21,285 posts
  • MVP

what I was wondering if it's possible to directly arrive to the 100,000 line by specifying the offset within the file (in bytes) or do i have to read 99,999 lines first ?

That depends. You can set it up to search the database for the specific line that you want.

The other thing is does each user have exclusive access to the file or can several users open and read the file at the same time ?

That depends on how you set it up, what OS the server is running, whether you go with SQL or flatfile. With SQL, everyone can access it at once.
  • 0

#7
rambleon

rambleon

    New Member

  • Topic Starter
  • Member
  • Pip
  • 3 posts
The location within the text file is in an sql database but the content is in the text file. I know the exact location, my question is if I can arrive directley to this location - like read 100,000 bytes in one go - or do I have to read the text line by line until I arrive at the location gived in the sql database?
  • 0






Similar Topics

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

As Featured On:

Microsoft Yahoo BBC MSN PC Magazine Washington Post HP