[rambleon]

Hi,
I have an asp.net intranet app. that allows employees to view their payslips for the last 5 years. The payslip info. is in a monthly flat text file. I have a database that contains the employees id number, the file name and the line number within the file where the employees payslip starts. The company has about 5000 employees, each file contains about 300,000 lines, for each payslip retrieved I have to open the file, read to the line of the employees payslip, present the payslip and close the file. In testing the performance is reasonable. My question is how will using this method in an intranet environment where many users are reading the same text file at the same time impact on performance and if anyone can suggest any alternative methods or ways to improve performance.

Thanks

Rambleon

[Advertisements]

hi there,
i don't know the answer to your question, but i think you should post this in a hardware forum (perhaps networking?) and you might get more help there

[TaNkZ101]

hi there,
i don't know the answer to your question, but i think you should post this in a hardware forum (perhaps networking?) and you might get more help there

[ScHwErV]

Since its ASP and database related, it belongs here.

Flatfiles are always going to be a slower idea than running an sql database. If your database was sql based, you could have it run queries and only display the necessary information versus all 300,000 lines.

Any time you run a single file that hundreds of people will access at the same time, you are asking for trouble.

ScHwErV

[rambleon]

The actual location of the text within the file is in an sql database. what I was wondering if it's possible to directly arrive to the 100,000 line by specifying the offset within the file (in bytes) or do i have to read 99,999 lines first ?
The other thing is does each user have exclusive access to the file or can several users open and read the file at the same time ?

[TDXkev]

If possible, you should lose the text file and put everything in your database. Otherwise, you're performance is not going to be that great if a lot of people are accessing the file at once, especially if you are reading in each line until you get to the one you want.

It might take time to input it into a database, but it will be worth it in the long run as it is much more maintainable than a flat file.

In summary:

1. The file I/O will hurt your performance.
2. The database queries would be much faster.
3. The database structure is a better idea for maintainability in the long run.
4. The database structure will be much easier to manipulate.

Good Luck!

[ScHwErV]

what I was wondering if it's possible to directly arrive to the 100,000 line by specifying the offset within the file (in bytes) or do i have to read 99,999 lines first ?

That depends. You can set it up to search the database for the specific line that you want.

The other thing is does each user have exclusive access to the file or can several users open and read the file at the same time ?

That depends on how you set it up, what OS the server is running, whether you go with SQL or flatfile. With SQL, everyone can access it at once.

[rambleon]

The location within the text file is in an sql database but the content is in the text file. I know the exact location, my question is if I can arrive directley to this location - like read 100,000 bytes in one go - or do I have to read the text line by line until I arrive at the location gived in the sql database?