Tuesday, March 27, 2012
Archiving
may not work if need to import binary data(not a fixed length column) with
quotes etc. What I am thinking is to export to another database and back it
up/archive on a regular basis. That way it will be easy to restore when
needed. Any other options?
Thanks
BVRVyas has a great article on his web site about archiving.
http://vyaskn.tripod.com/
"Uhway" <vbhadharla@.sbcglobal.net> wrote in message
news:OwmojqdKFHA.1476@.TK2MSFTNGP09.phx.gbl...
> What's the best way to archive tables with binary data? Flat
files/bcp/DTS
> may not work if need to import binary data(not a fixed length column) with
> quotes etc. What I am thinking is to export to another database and back
it
> up/archive on a regular basis. That way it will be easy to restore when
> needed. Any other options?
> Thanks
> BVR
>|||Here's the link: http://vyaskn.tripod.com/sql_archive_data.htm
--
HTH,
Vyas, MVP (SQL Server)
SQL Server Articles and Code Samples @. http://vyaskn.tripod.com/
"Uri Dimant" <urid@.iscar.co.il> wrote in message
news:Ogj%23SXfKFHA.3332@.TK2MSFTNGP15.phx.gbl...
Vyas has a great article on his web site about archiving.
http://vyaskn.tripod.com/
"Uhway" <vbhadharla@.sbcglobal.net> wrote in message
news:OwmojqdKFHA.1476@.TK2MSFTNGP09.phx.gbl...
> What's the best way to archive tables with binary data? Flat
files/bcp/DTS
> may not work if need to import binary data(not a fixed length column) with
> quotes etc. What I am thinking is to export to another database and back
it
> up/archive on a regular basis. That way it will be easy to restore when
> needed. Any other options?
> Thanks
> BVR
>|||Narayanan,
I saw your Link, it looks good but i think it may or may not work if the
orders for 6 months is more than 100,000.
What happens if the data is more than 100,000 as we have in a real-time
system.
my solution was (lets take 6 months period - 100,000 records)
If you have to leave 10,000 on live production database and remove the rest
to history database.
This is how I did and its working great.
1. Check the record count of the database based on date.
2. Use Chunking method - means copy 1000-5000 every time depending on the
load from the live database to temp archive database on the live server then
call another str proc to copy data from temp archive to remote archive
database server.
(reason being if network goes down btw live server/remote server [reason
unknown] you can still have data in temp not lost and can come back redo the
data.)
3. once copied to remote archive server, now come back delete that data from
the live database by "delete from live where seqno in temp archive database"
and later delete from temp archive. (seqno is a primary key field)
I go beyond this, once all the data has been copied, i check for the first
and last copied to archive server exists meaning it has successfully copied.
I wrote a program to do this, the program just calls 3 stored procedures and
str proc does all the above function.
Too much isn't it.
Arun
"Narayana Vyas Kondreddi" wrote:
> Here's the link: http://vyaskn.tripod.com/sql_archive_data.htm
> --
> HTH,
> Vyas, MVP (SQL Server)
> SQL Server Articles and Code Samples @. http://vyaskn.tripod.com/
>
> "Uri Dimant" <urid@.iscar.co.il> wrote in message
> news:Ogj%23SXfKFHA.3332@.TK2MSFTNGP15.phx.gbl...
>
> Vyas has a great article on his web site about archiving.
> http://vyaskn.tripod.com/
>
>
> "Uhway" <vbhadharla@.sbcglobal.net> wrote in message
> news:OwmojqdKFHA.1476@.TK2MSFTNGP09.phx.gbl...
> > What's the best way to archive tables with binary data? Flat
> files/bcp/DTS
> > may not work if need to import binary data(not a fixed length column) with
> > quotes etc. What I am thinking is to export to another database and back
> it
> > up/archive on a regular basis. That way it will be easy to restore when
> > needed. Any other options?
> >
> > Thanks
> > BVR
> >
> >
>
>
Thursday, March 22, 2012
architecture
The same place as was mention when you asked this question 4 times ago. :-)
"mat" wrote:
> where is the actual physical location of views, sp's and dts packages?
architecture
Views and stored procs are objects within a database. Their definitions are
stored in sysobjects, syscomments, and others system tables within their
database. Physical storage is in the data files.
DTS package definitions are stored in msdb. Look for system tables named
like 'sysdts%'. Physical storage will be in the msdb data file.
DTS packages
"mat" wrote:
> where is the actual physical location of views, sp's and dts packages?
sql
architecture
as far as I know, views and sps are part of your database file (.mdf) stored
as objects with the system tables. Typically a view is thought of as a
virtual table, or a stored query. The results of using a view are not
permanently stored in the database. The data accessed through a view is
actually constructed using standard T-SQL select command and can come from
one to many different base tables or even other views.
I hope this helps...
"mat" wrote:
> where is the actual physical location of views, sp's and dts packages?
architecture
ie where would the actual pysical location of vies and dts packages be
Views and stored procedures are in the syscomments table. DTS packages are
in the MSDB database.
Adam Machanic
SQL Server MVP
http://www.sqljunkies.com/weblog/amachanic
"mat" <mat@.discussions.microsoft.com> wrote in message
news:37BD3C41-73C3-4F81-AAFA-3E823244BBF4@.microsoft.com...
> Where are views, dts packages stored..
> ie where would the actual pysical location of vies and dts packages be
architecture
ie where would the actual pysical location of vies and dts packages beViews and stored procedures are in the syscomments table. DTS packages are
in the MSDB database.
Adam Machanic
SQL Server MVP
http://www.sqljunkies.com/weblog/amachanic
--
"mat" <mat@.discussions.microsoft.com> wrote in message
news:37BD3C41-73C3-4F81-AAFA-3E823244BBF4@.microsoft.com...
> Where are views, dts packages stored..
> ie where would the actual pysical location of vies and dts packages be
architecture
stored in sysobjects, syscomments, and others system tables within their
database. Physical storage is in the data files.
DTS package definitions are stored in msdb. Look for system tables named
like 'sysdts%'. Physical storage will be in the msdb data file.
DTS packages
"mat" wrote:
> where is the actual physical location of views, sp's and dts packages?
architecture
ie where would the actual pysical location of vies and dts packages beViews and stored procedures are in the syscomments table. DTS packages are
in the MSDB database.
Adam Machanic
SQL Server MVP
http://www.sqljunkies.com/weblog/amachanic
--
"mat" <mat@.discussions.microsoft.com> wrote in message
news:37BD3C41-73C3-4F81-AAFA-3E823244BBF4@.microsoft.com...
> Where are views, dts packages stored..
> ie where would the actual pysical location of vies and dts packages besql
architecture
stored in sysobjects, syscomments, and others system tables within their
database. Physical storage is in the data files.
DTS package definitions are stored in msdb. Look for system tables named
like 'sysdts%'. Physical storage will be in the msdb data file.
DTS packages
"mat" wrote:
> where is the actual physical location of views, sp's and dts packages?
architecture
as objects with the system tables. Typically a view is thought of as a
virtual table, or a stored query. The results of using a view are not
permanently stored in the database. The data accessed through a view is
actually constructed using standard T-SQL select command and can come from
one to many different base tables or even other views.
I hope this helps...
"mat" wrote:
> where is the actual physical location of views, sp's and dts packages?
architecture
"mat" wrote:
> where is the actual physical location of views, sp's and dts packages?
Monday, March 19, 2012
Applying of snapshot for transaction replication with DTS transfor
move all data from publisher to subscriber - apply snapshot to subscriber.
Subscriber have different schema but no data. How it is possible to do? I
expected snapshot will use same DTS, but it not use it. Publisher DB is 24/7
system.
Have a look at transformable subscriptions. I would also advise you to have
a look at custom sync objects and encapsulating the data mapping in the
replication stored procedures as opposed to use DTS transforms due to
performance reasons.
Here is an article explaining how to do this.
http://www.dbazine.com/sql/sql-artic...rm=replicating
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"andsm" <andsm@.discussions.microsoft.com> wrote in message
news:53CF05CA-FA93-4AAE-A4AA-37347B3060E7@.microsoft.com...
>I created transaction replication with DTS transformation. Next I want to
> move all data from publisher to subscriber - apply snapshot to subscriber.
> Subscriber have different schema but no data. How it is possible to do? I
> expected snapshot will use same DTS, but it not use it. Publisher DB is
> 24/7
> system.
Thursday, March 8, 2012
appling DTS custom transformation to replication problem (Urgent)
In my production environment, there is a source database that replicates
data to another server (using transactional replication).
Because of some upgrade needs, I need to change the subscriber database to
store Unicode. As a result, I want to customize the replication such that our
Big5 data in source database can be converted to UNICODE automatically during
replication.
Firstly, I want to know whether applying DTS custom transformation in
replication can solve my problem. Is there any better methods?
In addition, how can I configure DTS custom transforamtion.?
I have tried using Define Transformation of Published Data Wizard but error
occurs at the step of adding task of creating the DTS package. The error is
"SQL Server Enterprise Manager cannot complete this operation".
I also try to use DTS designer to create the package. But I cannot select
the package created bt DTS designer when I create a transformable
subscription.
Please help me
Thanks
"kazom" wrote:
> Hi all,
> In my production environment, there is a source database that replicates
> data to another server (using transactional replication).
> Because of some upgrade needs, I need to change the subscriber database to
> store Unicode. As a result, I want to customize the replication such that our
> Big5 data in source database can be converted to UNICODE automatically during
> replication.
> Firstly, I want to know whether applying DTS custom transformation in
> replication can solve my problem. Is there any better methods?
> In addition, how can I configure DTS custom transforamtion.?
> I have tried using Define Transformation of Published Data Wizard but error
> occurs at the step of adding task of creating the DTS package. The error is
> "SQL Server Enterprise Manager cannot complete this operation".
> I also try to use DTS designer to create the package. But I cannot select
> the package created bt DTS designer when I create a transformable
> subscription.
> Please help me
> Thanks
>
Hi all,
Let me make the question be more specific.
I just follow this document to configure the DTS custom transformation
replication.
But error occurs after step 11 of the part Define a DTS Package for the
Transformation
The error is:
SQL Server Enterprise Manager could not complete this operation
Both the publisher and subscripter is at the same server but different
database
The server is SQL Server 2000 service pack 4
Please help me. I really need your professional help
This is very urgent
Thanks
applied dts files to sql server
how do I write a script to apply this dts package to SQL server "server1"?
thanksWhy do you need to write a script?
Can't you just right-click the Data Transformation Services folder in
EM and Select Open Package.
You could then browse to the .dts file and do a save as and save on the
Server?
Barry|||but i have many dts files, I want to apply them automatically with scripts.
I know how to apply dts script manually.
"Barry" wrote:
> Why do you need to write a script?
> Can't you just right-click the Data Transformation Services folder in
> EM and Select Open Package.
> You could then browse to the .dts file and do a save as and save on the
> Server?
> Barry
>
Application times out while doing the updates
I am running a DTS to collect the summarized info from Oracle database
into SQL server. I then have a update job which updates my
transactional table from the summarized table.
The update takes a very long time (~ 3 minutes)even though it has
around 1500 rows which causes the application to timeout. I want this
job to be done in less than a minute.
Thoughts on improving performance. Is stored procedure a way to go?
(I have used Isolation,row hints etc etc..nothing seems to be working)
AJI suggest you review the execution plan of the UPDATE query. Perhaps
additional indexes or query changes will speed it up substantially. If you
need additional help, please post you table DDL (including constraints,
indexes and triggers) along with you query and sample data.
Simply encapsulating the query in a proc is unlikely to improve performance.
--
Hope this helps.
Dan Guzman
SQL Server MVP
"AJ" <aj70000@.hotmail.com> wrote in message
news:6097f505.0408272303.752889bc@.posting.google.c om...
> here's the scenario..
> I am running a DTS to collect the summarized info from Oracle database
> into SQL server. I then have a update job which updates my
> transactional table from the summarized table.
> The update takes a very long time (~ 3 minutes)even though it has
> around 1500 rows which causes the application to timeout. I want this
> job to be done in less than a minute.
> Thoughts on improving performance. Is stored procedure a way to go?
> (I have used Isolation,row hints etc etc..nothing seems to be working)
> AJ|||Thanks Dan,
Here's the entire structure and query
Table A has these columns
a_id(pkey),ftq,break_offs,completes,assigned,s_id, end_date (Indexed)
and few other columns
ftq,break_offs, completes,assigned are the summarized cols. which are
updated every hour from the summary_table.
Summary table has these cols.
a_id,ftq break_offs,completes,assigned.
I am issuing this update statement
UPDATE A SET
break_offs = (select case when break_offs<0 then 0 else break_offs end
break_offs from summary_table where A.ano = summary_table.ano),
ftq=(select ftq from summary_table where A.ano=summary_table.ano),
actually_assigned=(select member_assigned from summary_table where
A.ano=summary_table.ano),
qualified_completes=( select member_completes from summary_table where
A.ano=summary_table.ano)
WHERE EXISTS (select break_offs,ftq from summary_table where A.ano =
summary_table.ano) and
end_date>=getdate().
------------
Apart from this we also have stored procs, trigger on the same
table.But the thing is the jobs runs 15 minutes past the hour and
stalls everything.
Thanks again
AJ|||AJ (aj70000@.hotmail.com) writes:
> Here's the entire structure and query
> Table A has these columns
> a_id(pkey),ftq,break_offs,completes,assigned,s_id, end_date (Indexed)
> and few other columns
> ftq,break_offs, completes,assigned are the summarized cols. which are
> updated every hour from the summary_table.
> Summary table has these cols.
> a_id,ftq break_offs,completes,assigned.
Note here: Dan asked for the DDL. By this he means the CREATE TABLE
statements. These are useful if you want to test a query, and they
are also easier to read than a free-form list. In this case, the
cure appears simple enough anyway:
> UPDATE A SET
> break_offs = (select case when break_offs<0 then 0 else break_offs end
> break_offs from summary_table where A.ano = summary_table.ano),
> ftq=(select ftq from summary_table where A.ano=summary_table.ano),
> actually_assigned=(select member_assigned from summary_table where
> A.ano=summary_table.ano),
> qualified_completes=( select member_completes from summary_table where
> A.ano=summary_table.ano)
> WHERE EXISTS (select break_offs,ftq from summary_table where A.ano =
> summary_table.ano) and
> end_date>=getdate().
This should be faster:
UPDATE A
SET break_offs = case when s.break_offs < 0 then 0
else s.break_offs
end,
ftq = s.ftq,
actually_assigned = s.member_assigned,
qualified_completes = s.member_completes,
FROM A
JOIN summary_table s ON a.ano = s.ano
and A.end_date >= getdate()
This is uses syntax that is proprietary to MS SQL Server and Sybase,
but it is a lot more effecient, since in your version each subquery
is evaluated separately.
Permit me also to note that the last condition looks funky. getdate()
returns the current time, so if A.end_date is a date only, this condition
may not do what you expect. That is, rows where end_date = today will
not be updated.
--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se
Books Online for SQL Server SP3 at
http://www.microsoft.com/sql/techin.../2000/books.asp|||Hi Dan/Erland,
Is there a way to save the execution plan?..I ran trace/execution. My
update job seems to be firing at the end since I have lots of table
level triggers. The changes that Erland is suggesting is infact taking
longer than my query.
Thoughts/Comments
AJ
*** Sent via Developersdex http://www.developersdex.com ***
Don't just participate in USENET...get rewarded for it!|||AJ Jr (aj70000@.hotmail.com) writes:
> Is there a way to save the execution plan?..I ran trace/execution. My
> update job seems to be firing at the end since I have lots of table
> level triggers. The changes that Erland is suggesting is infact taking
> longer than my query.
You can run a query from Query Analyzer and press CTRL/K to get a graphical
showplan. Or you can press CTRL/L to get an estimated plan, so that the
query is not run. Tnis is not useful if you run the entire procedure, but
mainly if you run the troublesome statement separately.
If you have a Profiler trace with the Execution Plan event, you can find
the plan for the query, select that row, and the cut and paste from the
lower pane. Normally the plan you are looking for is the one before
StmtCompleted, but if there are triggers on the table, it may not be in
this case.
Too bad that my suggestion performed worse. That was indeed a bit of a
surprise. It would be interesting to see both plans.
It would help to have the complete CREATE TABLE and CREATE INDEX statements
for the tables.
--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se
Books Online for SQL Server SP3 at
http://www.microsoft.com/sql/techin.../2000/books.asp|||To add to Erland's response, you may want to add unique constraints or
indexes on the ano columns.
--
Hope this helps.
Dan Guzman
SQL Server MVP
"AJ" <aj70000@.hotmail.com> wrote in message
news:6097f505.0408281317.3e394ab2@.posting.google.c om...
> Thanks Dan,
>
> Here's the entire structure and query
> Table A has these columns
> a_id(pkey),ftq,break_offs,completes,assigned,s_id, end_date (Indexed)
> and few other columns
> ftq,break_offs, completes,assigned are the summarized cols. which are
> updated every hour from the summary_table.
> Summary table has these cols.
> a_id,ftq break_offs,completes,assigned.
> I am issuing this update statement
> UPDATE A SET
> break_offs = (select case when break_offs<0 then 0 else break_offs end
> break_offs from summary_table where A.ano = summary_table.ano),
> ftq=(select ftq from summary_table where A.ano=summary_table.ano),
> actually_assigned=(select member_assigned from summary_table where
> A.ano=summary_table.ano),
> qualified_completes=( select member_completes from summary_table where
> A.ano=summary_table.ano)
> WHERE EXISTS (select break_offs,ftq from summary_table where A.ano =
> summary_table.ano) and
> end_date>=getdate().
>
> ------------
> Apart from this we also have stored procs, trigger on the same
> table.But the thing is the jobs runs 15 minutes past the hour and
> stalls everything.
> Thanks again
> AJ|||Hi Dan/Erland
I have identified a problem : It is the Update trigger which has a
cursor which checks all the records before allowing the update.
Comments/Thoughts on this one :
The update job is the scheduled job..it runs as user x (I can specify
sa or DBO).
Is there a way in SQL server that I can fire the trigger on a table
for a particular user only (Application user).
Thanks ia advance
AJ|||AJ (aj70000@.hotmail.com) writes:
> I have identified a problem : It is the Update trigger which has a
> cursor which checks all the records before allowing the update.
> Comments/Thoughts on this one :
> The update job is the scheduled job..it runs as user x (I can specify
> sa or DBO).
> Is there a way in SQL server that I can fire the trigger on a table
> for a particular user only (Application user).
No, but you can use ALTER TRIGGER to disable the trigger. But this is
a little risky if the job dies for some reason in the middle of and leaves
the trigger disabled.
A better technique is to define a temp table, call it #triggerdisable, just
before the update, and then in the trigger add:
if object_id('tempdb..#triggerdisable') IS NOT NULL
RETURN
Note: what columns the table has or what data there is does not matter. It's
the sheer existence that matter.
But of course the best solution would be rewrite the trigger to use set-
based operations!
--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se
Books Online for SQL Server SP3 at
http://www.microsoft.com/sql/techin.../2000/books.asp|||Thanks Erland,
But I am kinda lost...By defining #triggerdisable..What do you mean?
Thanks again
AJ|||AJ (aj70000@.hotmail.com) writes:
> But I am kinda lost...By defining #triggerdisable..What do you mean?
CREATE TABLE #triggerdisable(a int NOT NULL)
The temp table serves as a process-global flag to test for.
--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se
Books Online for SQL Server SP3 at
http://www.microsoft.com/sql/techin.../2000/books.asp|||thanks Erland/Dan for all your help.
AJ
Saturday, February 25, 2012
application name in profiler, could it be modifed
We have 500+ dts running 24 X 7 ,started from dts run
In profiler every enty in application name column show as Dts
designer.
1.
Is possible to modify dts property or anything and add customer
informatiom ?
example
application name = dtsrun - Insert_Into_table_X
2.
is possible to modify application name for any connection ?
Thanks
AlexSure. For a SQL connection just bring up the properties and click on the
Advanced button and you'll see an Application Name property. You could for
example add an ActiveX script task to the start of your package that loops
through the SQL connections and sets the Application Name property of the
connection to the package name.
--
HTH,
Jasper Smith (SQL Server MVP)
http://www.sqldbatips.com
<t2581@.hotmail.com> wrote in message
news:1137770497.717723.119880@.g14g2000cwa.googlegroups.com...
> Hi,
>
> We have 500+ dts running 24 X 7 ,started from dts run
> In profiler every enty in application name column show as Dts
> designer.
>
> 1.
> Is possible to modify dts property or anything and add customer
> informatiom ?
> example
> application name = dtsrun - Insert_Into_table_X
> 2.
> is possible to modify application name for any connection ?
>
> Thanks
>
> Alex
>
application name in profiler, could it be modifed
We have 500+ dts running 24 X 7 ,started from dts run
In profiler every enty in application name column show as Dts
designer.
1.
Is possible to modify dts property or anything and add customer
informatiom ?
example
application name = dtsrun - Insert_Into_table_X
2.
is possible to modify application name for any connection ?
Thanks
Alex
Sure. For a SQL connection just bring up the properties and click on the
Advanced button and you'll see an Application Name property. You could for
example add an ActiveX script task to the start of your package that loops
through the SQL connections and sets the Application Name property of the
connection to the package name.
HTH,
Jasper Smith (SQL Server MVP)
http://www.sqldbatips.com
<t2581@.hotmail.com> wrote in message
news:1137770497.717723.119880@.g14g2000cwa.googlegr oups.com...
> Hi,
>
> We have 500+ dts running 24 X 7 ,started from dts run
> In profiler every enty in application name column show as Dts
> designer.
>
> 1.
> Is possible to modify dts property or anything and add customer
> informatiom ?
> example
> application name = dtsrun - Insert_Into_table_X
> 2.
> is possible to modify application name for any connection ?
>
> Thanks
>
> Alex
>
application name in profiler, could it be modifed
We have 500+ dts running 24 X 7 ,started from dts run
In profiler every enty in application name column show as Dts
designer.
1.
Is possible to modify dts property or anything and add customer
informatiom ?
example
application name = dtsrun - Insert_Into_table_X
2.
is possible to modify application name for any connection ?
Thanks
AlexSure. For a SQL connection just bring up the properties and click on the
Advanced button and you'll see an Application Name property. You could for
example add an ActiveX script task to the start of your package that loops
through the SQL connections and sets the Application Name property of the
connection to the package name.
HTH,
Jasper Smith (SQL Server MVP)
http://www.sqldbatips.com
<t2581@.hotmail.com> wrote in message
news:1137770497.717723.119880@.g14g2000cwa.googlegroups.com...
> Hi,
>
> We have 500+ dts running 24 X 7 ,started from dts run
> In profiler every enty in application name column show as Dts
> designer.
>
> 1.
> Is possible to modify dts property or anything and add customer
> informatiom ?
> example
> application name = dtsrun - Insert_Into_table_X
> 2.
> is possible to modify application name for any connection ?
>
> Thanks
>
> Alex
>
Friday, February 24, 2012
appending to the end of current file
ThanksDarren Green suggested in earlier posts that :
The text file provider does not support this. The simplest thing to do is to export to a new file, then use the DOS copy command to merge the files.
This can be done with the Execute Process Task, and any dynamic filename stuff can be handled with an ActiveX Script Task to change the connection and the Exec Proc Task at the same time.
copy FileOriginal.txt+FileNew.txt FileResult.txt
HTH|||That is what I am currently doing. I was hoping for something within the dts package. Thanks for you help.
val|||As suggested its not available using TEXT provider with DTS.
May be this is also other way around, is to export to an excel sheet and save that as text file.:)
Appending to a text file via DTS
Kindly let me know where I am going wrong.I am having the same issue as well... I thought I remember seeing an option checkbox somewhere to do that... but I cant seem to find it.
I doubt we are the only ones having to deal with this, so can someone please chime in with an answer or maybe a possible direction for us to follow? Thanks in advance... :)