Wednesday, March 7, 2012

Data Archiving

If I have a huge database, I use the data archiving to archieve to table. eg. I have table A, a huge table, I archive some of the old data to table B. Let said my data is some transaction data. When I would like to retrieve it to generate a report, a historical report for my yearly transactions. Let said I've split the March 2003 data from table A into table B. which mean only April to July transactions remain in table A. But now I want to retrieve the data from Jannuary till now on, to generate a half year report. How should I do it?? Should I just using a SQL query to select data from table A where date from Jan to July, then the system will automatically look back for the data that have been archived into table B and retrieve it for me?? Or I should use 2 seperate queries to select it from table A (Apr-July) and table B (Jan-Mar)?esupport,

you could create a view like the following:

create View MyData as
select * from MyOldData
union all
select * from MyActualData
go

and to query from this view instead of querying from thge tables.|||If using View instead of querying from table, will this affect my performance or my database size? will it slowing down my system performance? Because normally my database will be quite big in size. 2-3 GB. Is this the optimum solution to solve the problem?|||Your other alternative is to maintain your current stragey (if it's working) and possibly create an aggregate table. This will only work in some circumstances, it depends on the needs of your business.

For instance if table A and B contains individual transactions (lets say 100,000 a day or something). However your reports only ever report at a daily level, then once a day summerise the data:

select date, sum(sales)
from table A
group by date

Load that summerised data into your aggregate table and you should be good to go..

This would cut down the table size of the table you are reporting from by n rows a day and still maintain all your transactions in your other table...

This is sort of moving into the data warehousing concepts and it may not work for you, but I'd just thought I'd suggest it..

Good luck.|||Thanks for the suggestions. It's help a lot.

BTW, is there any tools or built-in command to archive data from my production database into a history archive database in SQL Server 2000?|||Depending on how complex you want to get it might be possible to get away with just a stored procedure or 2... Otherwise the DTS tool might come in handy and if you want to get really serious, then Analysis Server if you have the licence and is appropriate etc...

Have Fun..|||If I'm using the UNION ALL suggested by kukuk

create View MyData as
select * from MyCurrentData
union all
select * from MyHistoryData
go

Let said I've archived my database till March into table MyHistoryData. Now I would like to retrieve some data for reporting purpose. I want to retrieve data from June to July, how would be the performance? How would the query performing? Will it "search" through all the data in table MyCurrentData and MyHistoryData?

How's the performing comparison by using a single query "SELECT * FROM MyCurrentData" compare with using UNION ALL as the example given above to retrieve data from 2 tables?|||Hi,

I think, BOL will answer your questions.
Look there for "Using Partitioned Views"|||Another question from me.
What is the size of the database table after achiving?
Will it become smaller or same size?
Let said my database before archive is 2GB, after archive, willl it become smaller? less than 2GB?|||Any ideas on back-end processing?
Lets say I'm doing retrieving data history for reporting. In the mean time, I would like to continue with my front-end processing, such as daily transactions?
In my current system, while I'm doing my data retrieve, I cannot do anything before the retrieving process finished. One of the issue I think is because of the size of my database and the technic that I'm using to retrieve the data is bad. Hope to hear more advices.

No comments:

Post a Comment