Wednesday, April 11, 2012

Script Task in SSIS

Abstract
This article demonstrates creating a SQL Server Integration Services package that imports a text file into SQL Server database table using a Script Task component.
Requirements
Article
We begin by launching Microsoft Visual Studio and create a new Integration Services Project which is located under Business Intelligence Projects category. After you have assigned a project name, proceed to click and drag the Script Task in Control Flow pane of the package's toolbox. Right click the script task and click on "Edit" Under the Script Task Editor change the "ScriptLanguage" to "Microsoft Visual C# 2008".
In Project Explorer, ensure the following references are added:
Back to the code window, ensure that the following namespaces are declared:
After the above declarations, proceed to creating a new application instance:
Application selectSIFISO_app = new Application();
Create package:
Package sS_pkg = new Package();
Assign relevant package name and description:
sS_pkg.Name = "Load Flat File Source into OLE DB Destination Using C#";
sS_pkg.Description = "Programmatically create an SSIS 2008 package that loads a Flat File Source into OLE DB Destination Using Script Task's C# language";
Insert the Data Flow Task with appropriate name and some buffer space for processing of file (the last part is optional - you can also use default buffer allocation):
sS_pkg.Executables.Add("STOCK:PipelineTask");
TaskHost taskHost = sS_pkg.Executables[0] as TaskHost;
MainPipe dataFlowTask = (MainPipe)taskHost.InnerObject;
taskHost.Name = "Dynamic Data Flow Task";
taskHost.Properties["DefaultBufferMaxRows"].SetValue(taskHost, "1000000");
Insert the Flat File connection:
ConnectionManager connectionManagerFlatFile = sS_pkg.Connections.Add("FLATFILE");
You can change this path depending on where you have stored the flat file (ensure you download the attached file, see "Requirements" section above):
connectionManagerFlatFile.ConnectionString = @"C:\Temp\flat_src.txt";
Assign name to the flat file connection:
connectionManagerFlatFile.Name = "TXT_FlatFile";
Indicate that the flat file is delimited:
connectionManagerFlatFile.Properties["Format"].SetValue(connectionManagerFlatFile, "Delimited");
Indicate whether the source file has column headings or not - in this case, our sample data has column headings, hence - true:
connectionManagerFlatFile.Properties["ColumnNamesInFirstDataRow"].SetValue(connectionManagerFlatFile, Convert.ToBoolean(true));
Get native Flat File connection:
RuntimeWrapper.IDTSConnectionManagerFlatFile100 connectionFlatFile = connectionManagerFlatFile.InnerObject as RuntimeWrapper.IDTSConnectionManagerFlatFile100;
Declare local string variable that will be used as part of reading the text file:
string line;
Determine the number of columns by reading the sample Flat File - line by line:

using (StreamReader file = new StreamReader(@"C:\Temp\flat_src.txt"))
{
 try
    {
    while ((line = file.ReadLine()) != null)
   {
     char[] delimiters = new char[] { '|' };
     string[] parts = line.Split(delimiters, StringSplitOptions.RemoveEmptyEntries);
     
        for (int i = 0; i < parts.Length; i++)
 {
    RuntimeWrapper.IDTSConnectionManagerFlatFileColumn100 flatFileCol 
    = connectionFlatFile.Columns.Add() as RuntimeWrapper.IDTSConnectionManagerFlatFileColumn100;
    sS_AssignColumnProperties(flatFileCol, parts[i], "|");
 }
     //Exit file after reading the first line
     break;
   }                
  }
 catch (Exception ex)
    {
     throw ex;
    }
 finally
    {
     file.Close();
    }
}
Edit the last Flat File column delimiter into NewLine instead of a Comma:
connectionFlatFile.Columns[connectionFlatFile.Columns.Count - 1].ColumnDelimiter = Environment.NewLine;
Insert Flat File source component:
IDTSComponentMetaData100 componentSource = dataFlowTask.ComponentMetaDataCollection.New();
componentSource.Name = "FlatFileSource";
componentSource.ComponentClassID = "DTSAdapter.FlatFileSource";
Insert source design-time instance and initialise component:
CManagedComponentWrapper instanceSource = componentSource.Instantiate();
instanceSource.ProvideComponentProperties();
Set source connection:
componentSource.RuntimeConnectionCollection[0].ConnectionManagerID = connectionManagerFlatFile.ID; componentSource.RuntimeConnectionCollection[0].ConnectionManager = DtsConvert.ToConnectionManager90(connectionManagerFlatFile);
Reinitialize Flat File source metadata:
instanceSource.AcquireConnections(null);
instanceSource.ReinitializeMetaData();
instanceSource.ReleaseConnections();
Insert the SQL Server 2008 OLE-DB connection:
ConnectionManager connectionManagerOleDb = sS_pkg.Connections.Add("OLEDB");
connectionManagerOleDb.ConnectionString = string.Format("Provider=SQLOLEDB.1;Data Source={0};Initial Catalog={1};Integrated Security=SSPI;", "localhost", "AdventureWorks");
connectionManagerOleDb.Name = "OLEDB";
connectionManagerOleDb.Description = "OLEDB Connection";
Insert OLE-DB destination:
IDTSComponentMetaData100 componentDestination = dataFlowTask.ComponentMetaDataCollection.New(); componentDestination.Name = "OLEDBDestination";
componentDestination.Description = "OLEDB Destination for the Flat File data load";
componentDestination.ComponentClassID = "DTSAdapter.OLEDBDestination";
Insert destination design-time instance and initialise component:
CManagedComponentWrapper instanceDestination = componentDestination.Instantiate(); instanceDestination.ProvideComponentProperties();
Set destination connection:
componentDestination.RuntimeConnectionCollection[0].ConnectionManagerID = connectionManagerOleDb.ID; componentDestination.RuntimeConnectionCollection[0].ConnectionManager = DtsConvert.ToConnectionManager90(connectionManagerOleDb);
Indicates the name of the database object used to open a rowset: instanceDestination.SetComponentProperty("OpenRowset", "[dbo].[sS_flatfileLoad]");
Specifies the mode used to open the database:
instanceDestination.SetComponentProperty("AccessMode", 3);
Specifies options to be used with fast load. Applies only if fast load is turned on: instanceDestination.SetComponentProperty("FastLoadOptions", "TABLOCK,CHECK_CONSTRAINTS");
Indicates whether the values supplied for identity columns will be copied to the destination or not In this case, we have set this property to false:
instanceDestination.SetComponentProperty("FastLoadKeepIdentity", false);
Indicates whether the columns containing null willhave null inserted in the destination or not In this case, we have opted no to insert nulls:
instanceDestination.SetComponentProperty("FastLoadKeepNulls", false);
Specifies the column code page to use when code page information is unavailable from the data source In this case we used the default - 1252:
instanceDestination.SetComponentProperty("DefaultCodePage", 1252);
Specifies when commits are issued during data insertion In this case, we have opted for the default size which is set to 2147483647:
instanceDestination.SetComponentProperty("FastLoadMaxInsertCommitSize", 2147483647);
Indicates the number of seconds before a command times out In this case, we have opted for the default value of 0 which indicates an infinite time-out:
instanceDestination.SetComponentProperty("CommandTimeout", 0);
Indicates the usage of DefaultCodePage property value when describing the character data In this case, we have opted for the default value of false:
instanceDestination.SetComponentProperty("AlwaysUseDefaultCodePage", false);
Connect the Flat File source to the OLE DB Destination component: dataFlowTask.PathCollection.New().AttachPathAndPropagateNotifications(componentSource.OutputCollection[0] ,componentDestination.InputCollection[0]);
Get input and virtual input for destination to select and map columns:
IDTSInput100 destinationInput = componentDestination.InputCollection[0];
IDTSVirtualInput100 destinationVirtualInput = destinationInput.GetVirtualInput();
IDTSVirtualInputColumnCollection100 destinationVirtualInputColumns = destinationVirtualInput.VirtualInputColumnCollection;
Reinitialize the metadata, generating exernal columns from flat file columns:
instanceDestination.AcquireConnections(null);
instanceDestination.ReinitializeMetaData();
instanceDestination.ReleaseConnections();
Select and map destination columns:
foreach (IDTSVirtualInputColumn100 virtualInputColumn in destinationVirtualInputColumns)
{ // Select column, and retain new input column
IDTSInputColumn100 inputColumn = instanceDestination.SetUsageType(destinationInput.ID,destinationVirtualInput, virtualInputColumn.LineageID, DTSUsageType.UT_READONLY);
// Find external column by name
IDTSExternalMetadataColumn100 externalColumn = destinationInput.ExternalMetadataColumnCollection[inputColumn.Name];
// Map input column to external column
instanceDestination.MapInputColumn(destinationInput.ID, inputColumn.ID, externalColumn.ID);
}
Execute the package or disable the below code if you intend running the package later:
sS_pkg.Execute();
Finally, save the package - in this case, we have opted to save the package into file system:
selectSIFISO_app.SaveToXml(@"E:\newArticle.dtsx", sS_pkg, null); Dts.TaskResult = (int)ScriptResults.Success;
In addition to the above code, you will notice that some part of the code references to the below function. This function is used to assign DTS column properties:
private static void sS_AssignColumnProperties(RuntimeWrapper.IDTSConnectionManagerFlatFileColumn100 flatFileCol, string getColName, string getDelim) {
Assign delimiter:
flatFileCol.ColumnType = "Delimited";
flatFileCol.ColumnDelimiter = getDelim;
Indicate column data type - in this case, all the source columns will be set to String Data Type:
flatFileCol.DataType = RuntimeWrapper.DataType.DT_STR;
Indicate column width - in this case, width of all source columns will be set to a length of 100:
flatFileCol.ColumnWidth = 100;
Assign column name:
RuntimeWrapper.IDTSName100 columnName = flatFileCol as RuntimeWrapper.IDTSName100; columnName.Name = getColName.ToString(); 

Execute the ssis multiple ssis package using the T-Sql

If you want to execute a set of SSIS packages in SQL Server 2008 or 2005, you can do this using T-SQL. First you will  need a table with all of your package names on it. Then a While loop to execute each package.
Here is the example code:

Declare @FilePath varchar(2000)
Declare @cmd varchar(2000)
 
DECLARE @package_name varchar(200)
Declare @PackageCount int
Declare @X int
Set @X = 1
Set @PackageCount = (Select COUNT(*) from Packages)
set @FilePath = 'C:\Package Path'
While (@X <= @PackageCount)
Begin
 
    With PackageList as
    (
    Select PackageName, Row_Number() Over(Order by PackageName) as  Rownum
    From Packages
    )
    SELECT @package_name = PackageName
    FROM PackageList
    Where Rownum = @X
 
    select @cmd = 'DTExec /F "' + @FilePath + @Package_name + '"'
 
    print @cmd
   
    Set @X = @X + 1
   
    exec master..xp_cmdshell @cmd
 
 
End
In the new version of SSIS 2012 you will be able to launch packages with T-SQL Natively.

Thursday, February 16, 2012

How to Protect the Folder


  1. Create a new folder and name it whatever you would like.
  2. Open the folder, right-click on a blank area in it, then select New -> Text Document from the pop-up menu.
  3. Open the text file you just created by double-clicking it and copy/paste in the following text:
    cls
    @ECHO OFF
    title Folder Private
    if EXIST "Control Panel.{21EC2020-3AEA-1069-A2DD-08002B30309D}" goto UNLOCK
    if NOT EXIST Private goto MDLOCKER
    :CONFIRM
    echo Are you sure you want to lock the folder(Y/N)
    set/p "cho=>"
    if %cho%==Y goto LOCK
    if %cho%==y goto LOCK
    if %cho%==n goto END
    if %cho%==N goto END
    echo Invalid choice.
    goto CONFIRM
    :LOCK
    ren Private "Control Panel.{21EC2020-3AEA-1069-A2DD-08002B30309D}"
    attrib +h +s "Control Panel.{21EC2020-3AEA-1069-A2DD-08002B30309D}"
    echo Folder locked
    goto End
    :UNLOCK
    echo Enter password to unlock folder
    set/p "pass=>"
    if NOT %pass%== PASSWORD_GOES_HERE goto FAIL
    attrib -h -s "Control Panel.{21EC2020-3AEA-1069-A2DD-08002B30309D}"
    ren "Control Panel.{21EC2020-3AEA-1069-A2DD-08002B30309D}" Private
    echo Folder Unlocked successfully
    goto End
    :FAIL
    echo Invalid password
    goto end
    :MDLOCKER
    md Private
    echo Private created successfully
    goto End
    :End
  4. In the above code, replace the key PASSWORD_GOES_HERE with the password you want to use to unlock the folder. For example if you want the password to be123456, the line should look like:
    if NOT %pass%== 123456 goto FAIL
  5. Save your new file in the .bat format with the complete file name being locker.bat. To do this, make sure to change the Save as type: to All Files (*.*).
  6. In the folder you created back in Step #1, double click the locker.bat file and there will now be a new folder named Private where you can put anything you want.
  7. Upon exiting, double click the locker.bat file again. It will prompt you to answer whether you want to lock your folder or not. Press Y and the private folder will disappear.
  8. In order to retrieve the Private folder, all you have to do is double click thelocker.bat file and enter the password which you set in Step #4 and the folder will appear again for you to access.
  9. That’s it!

Wednesday, January 25, 2012

Important Question


differenct b/w the subquery and join

why we use stored proc

filter index

staging server

Bcp-Bulk copy program (
links

http://www.codeproject.com/Articles/16922/SQL-Bulk-Copy-with-C-Net
http://blog.sqlauthority.com/2008/02/06/sql-server-import-csv-file-into-sql-server-using-bulk-insert-load-comma-delimited-file-into-sql-server/
http://msdn.microsoft.com/en-us/library/aa196743(v=sql.80).aspx
http://www.dotnetcurry.com/ShowArticle.aspx?ID=323
http://www.sqlteam.com/article/using-bulk-insert-to-load-a-text-file
)




**************************


Inside the function we can't handle the
exception and can't use the transactions.

by using the select statement we can use the function.

Inside the function we can't write the insert or update command.

Inside the function we can't call the stored Procedure.

Inside the Procedure the exception handling and transaction .
Procedure.




Monday, January 23, 2012

Difference Between Variable and Property:


Variables and Propertires both represent values that you can access.However, there are differences in storage and implementation.

Variables: A variable corresponds directly to a memory location. you define a variable with a single declaration statement. A variable can be a local variable, defined inside a procedure and available only within that procedure,
or it can be a member variable, defined in a module, class, or structure but not inside any procedure .
a member variable is also called a field.

Properties: A property is a data element defined on a module, class, or structure. you define a property with a code block between the Property and End Property Statements.
The code block contain the Get Procedure, a Set procedure, or both.
These procedures are called property procedures or Property accessors.



*****

Imp Difference B/W the Variable and Properties:

Point of Difference Variable Property

1. Declaration single declaration series of statements in a code block statement

2. Implementation Single storage      Executable code (property procedures)
                              location

3. storage Directly associated    typically has internal storage not available outside the
                         with variables           property's conaining class or module with variable's.
                                                        value property's value might or might  not exist as a stored element.
                     

4. Executable code none Must have at least one procedure

5. Read and write access read/Write or read only                 read/write, read-only, or write only

6.Custom actions (in addition  not possible )  can be performed as part of setting or
                                                                 retrieving  property value.to acception or returning value)




Examples:::



public class Car
{

    int speed; //Is this sufficient enough if Car will only set and get it.

    public Car(int initialSpeed)
    {
        speed = initialSpeed;
    }

    //Is this actually necessary, is it only for setting and getting the member
        //variable or does it add some benefit to it, such as caching and if so,
        //how does caching work with properties.
    public int Speed 
    {
        get{return speed;}
        set{speed = value;}
    }

        //Which is better?
        public void MultiplySpeed(int multiply)
        {
            speed = speed * multiply; //Line 1
            this.Speed = this.Speed * multiply; //Line 2

            //Change speed value many times
            speed = speed + speed + speed;
            speed = speed * speed;
            speed = speed / 3;
            speed = speed - 4;

        }
}




1 - Fields can’t be used in Interfaces
You can’t enforce the existence of a field in an object’s public contract through an interface. For properties though it works fine.
2 - Validation
While your application currently may not require any validation logic to set a particular value, changing business requirements may require inserting this logic later. At that point changing a field to a property is a breaking change for consumers of your API. (For example if someone was inspecting your class via reflection).
3 - Binary Serialization
Changing a field to a property is a breaking change if you’re using binary serialization. Incidentally, this is one of the reasons VB10’s auto-implemented properties have a “bindable” backing field (i.e. you can express the name of the backing field in code) – that way, if you change an auto-implemented property to an expanded property, you can still maintain serialization compatibility by keeping the backing field name the same (in C# you’re forced to change it because it generates backing fields with unbindable names).
4 - A lot of the .NET databinding infrastructure binds to properties but not fields
I’ve heard arguments on both sides as to whether or not that’s a good thing, but the reality is that’s the way it works right now. (Note from me: WPF bindings work on properties)
5 - Exposing a public field is an FxCop violation
For many of the reasons listed above :)
http://www.codinghorror.com/blog/2006/08/properties-vs-public-variables.html
http://blogs.msdn.com/b/vbteam/archive/2009/09/04/properties-vs-fields-why-does-it-matter-jonathan-aneja.aspx

Qualcomm Short Term

  113 is the SL. 1st Target by mid July.

Total Pageviews