data cleansing and translation tools for relational databases

Hello All
We have a data migration project. This is basically what we plan to
achieve we have a legacy system which as some data with German long
texts like material texts,etc
We want to take data coming in flat files which as this german texts
and convert this to English text the texts are primarily products
names please advise what tools external or whatever is available to do
this translation to english the texts are about 15 to 20 characters
max.
Next we have data coming from legacy systems that we want to cleanse
like identify duplicates based on things like addresses:
eq: we might have a customer called ABC one at address 118 Main
Street, Nashua and another same customer defined in our another legacy
system called ABC Inc but address at Main Street, Nashua
We want a data cleansing tool or library that works on top of SQL
Server that can do this kind of data pattern identification,etc
Please advice free tools and tools within SQL if I am not aware of
that can do this as well good 3rd party tools that can do this.
Thanks
KarenHi
I don't know of any data cleansing tools that may perform all the functions
that you require. Even if you employed a company to manually clean the data
there is likely to be some degree of error at th e nd of the exercise.
I you carried out the changes yourself it may be possible to clean the data
using a translation table and/or ad-hoc queries.
John
"Karen Middleton" <karenmiddleol@.yahoo.com> wrote in message
news:a5fd468a.0409032125.35235ea@.posting.google.com...
> Hello All
> We have a data migration project. This is basically what we plan to
> achieve we have a legacy system which as some data with German long
> texts like material texts,etc
> We want to take data coming in flat files which as this german texts
> and convert this to English text the texts are primarily products
> names please advise what tools external or whatever is available to do
> this translation to english the texts are about 15 to 20 characters
> max.
> Next we have data coming from legacy systems that we want to cleanse
> like identify duplicates based on things like addresses:
> eq: we might have a customer called ABC one at address 118 Main
> Street, Nashua and another same customer defined in our another legacy
> system called ABC Inc but address at Main Street, Nashua
> We want a data cleansing tool or library that works on top of SQL
> Server that can do this kind of data pattern identification,etc
> Please advice free tools and tools within SQL if I am not aware of
> that can do this as well good 3rd party tools that can do this.
> Thanks
> Karen|||There are many companies providing software and services to help clean up
name and address databases. These methods are usually specific to particular
postal systems or geographical areas so I suggest you Google for something
that meets your requirements in the locales of interest to you.
--
David Portas
SQL Server MVP
--|||Try the following link :
http://www.itcg.nl/
From their site :
CLUE
Have you ever experienced problems with duplicate customers,
addresses, products or any other information?
CLUE®, the CLUster Engine from ITCG is a generic component that uses
fuzzy matching logic to match data that is nearly the same. As a generic
component CLUE® is useful in many ways.
Few years ago they were specialising in cleaning data and
merging data from several sources into one database.
They had some fancy algoritms to detect double entries in
the database. They were very convincing in their presentation,
but I have no experience with their range of products.
good luck,
ben brugman
"Karen Middleton" <karenmiddleol@.yahoo.com> wrote in message
news:a5fd468a.0409032125.35235ea@.posting.google.com...
> Hello All
> We have a data migration project. This is basically what we plan to
> achieve we have a legacy system which as some data with German long
> texts like material texts,etc
> We want to take data coming in flat files which as this german texts
> and convert this to English text the texts are primarily products
> names please advise what tools external or whatever is available to do
> this translation to english the texts are about 15 to 20 characters
> max.
> Next we have data coming from legacy systems that we want to cleanse
> like identify duplicates based on things like addresses:
> eq: we might have a customer called ABC one at address 118 Main
> Street, Nashua and another same customer defined in our another legacy
> system called ABC Inc but address at Main Street, Nashua
> We want a data cleansing tool or library that works on top of SQL
> Server that can do this kind of data pattern identification,etc
> Please advice free tools and tools within SQL if I am not aware of
> that can do this as well good 3rd party tools that can do this.
> Thanks
> Karen|||Another thing you might look at for address cleaning is CASS certification.
This is a standard set by the US Postal service, and there is much software
around to help identify bad addresses, but it does not address your other
data quality problems.
--
Wayne Snyder, MCDBA, SQL Server MVP
Mariner, Charlotte, NC
www.mariner-usa.com
(Please respond only to the newsgroups.)
I support the Professional Association of SQL Server (PASS) and it's
community of SQL Server professionals.
www.sqlpass.org
"Karen Middleton" <karenmiddleol@.yahoo.com> wrote in message
news:a5fd468a.0409032125.35235ea@.posting.google.com...
> Hello All
> We have a data migration project. This is basically what we plan to
> achieve we have a legacy system which as some data with German long
> texts like material texts,etc
> We want to take data coming in flat files which as this german texts
> and convert this to English text the texts are primarily products
> names please advise what tools external or whatever is available to do
> this translation to english the texts are about 15 to 20 characters
> max.
> Next we have data coming from legacy systems that we want to cleanse
> like identify duplicates based on things like addresses:
> eq: we might have a customer called ABC one at address 118 Main
> Street, Nashua and another same customer defined in our another legacy
> system called ABC Inc but address at Main Street, Nashua
> We want a data cleansing tool or library that works on top of SQL
> Server that can do this kind of data pattern identification,etc
> Please advice free tools and tools within SQL if I am not aware of
> that can do this as well good 3rd party tools that can do this.
> Thanks
> Karen|||I would suggest calling a company like www.listadvantage.com and have
the names cleaned and cass certified.
they r self serve and very easy to work with
"ben brugman" <ben@.niethier.nl> wrote in message news:<OlCP4zAlEHA.536@.TK2MSFTNGP11.phx.gbl>...
> Try the following link :
> http://www.itcg.nl/
> From their site :
> CLUE
> Have you ever experienced problems with duplicate customers,
> addresses, products or any other information?
> CLUE®, the CLUster Engine from ITCG is a generic component that uses
> fuzzy matching logic to match data that is nearly the same. As a generic
> component CLUE® is useful in many ways.
>
> Few years ago they were specialising in cleaning data and
> merging data from several sources into one database.
> They had some fancy algoritms to detect double entries in
> the database. They were very convincing in their presentation,
> but I have no experience with their range of products.
> good luck,
> ben brugman
>
> "Karen Middleton" <karenmiddleol@.yahoo.com> wrote in message
> news:a5fd468a.0409032125.35235ea@.posting.google.com...
> > Hello All
> >
> > We have a data migration project. This is basically what we plan to
> > achieve we have a legacy system which as some data with German long
> > texts like material texts,etc
> >
> > We want to take data coming in flat files which as this german texts
> > and convert this to English text the texts are primarily products
> > names please advise what tools external or whatever is available to do
> > this translation to english the texts are about 15 to 20 characters
> > max.
> >
> > Next we have data coming from legacy systems that we want to cleanse
> > like identify duplicates based on things like addresses:
> >
> > eq: we might have a customer called ABC one at address 118 Main
> > Street, Nashua and another same customer defined in our another legacy
> > system called ABC Inc but address at Main Street, Nashua
> >
> > We want a data cleansing tool or library that works on top of SQL
> > Server that can do this kind of data pattern identification,etc
> >
> > Please advice free tools and tools within SQL if I am not aware of
> > that can do this as well good 3rd party tools that can do this.
> >
> > Thanks
> > Karen

Data Change

Can anybody suggest the best way I can achieve the following.
To select anybody whose surname has change in the last week, and to automatically flag a code field with "C". :cool:Hi,

Does your table have date field which is updated everytime a user updates the table?|||Yes I have found that it has a Modifcation Date field.|||Is there anyway to tell whether it is the surname that has been updated? Or could it be another field?|||Are you looking for a general solution, or is this a one-time issue?

For a general solution, you should use a trigger to stamp a datefield anytime a surname is modified.

For a one-time solution, restore a backup of your table from a week ago under a different name, and then join them on their primary keys to compare surnames.

....you do have weekly backups, right?|||if you have no record of this information,(the prior updating of the surname) you will need to create a audit column or and audit table then you can create a trigger that will indicate as such from here on out.

here are some {books online} articles you might want to look at.

Create Trigger
Alter Trigger
Programming Triggers

Data Archiving...sounds easy but how to approach.

hell all,
What is the best way to achieve the following mechanism.

My product has a database, which will be installed at my client site.
as we all know database is something which will grow tremendously.
Now i am planning to come out with a Data archiving facility in such a way that
Yearly data will be backed up and will be maintained in a separate machine.
The data that was backed up will be removed from the current machine's database.Ofcourse
there will be a mchanism provided through which i can transfer data from offline database to online database.
Now, please throw u r thoughts on how to approach this problem...
My database is in SQL Server.

Thanks and regards
saiHi Sai -

I don't know if this is the best way to accomplish this task -

but i set up a job that runs each day at 4:30am (after backup is completed)

that simply deletes all records that are 365 days old in the tables
that i want to keep trimmed -

I do have a datetime stamp on each table that is populated when the record is added to the table -

hope this helps -

take care
tony