Skip to main content

Posts

Extracting Data in a Database Using Python

Extracting data in a database using python. Using Python to extract data in a MySQL table first we need to pip install pymysql Create table along with connection to the mysql server using python import pymysql connection=pymysql.connect(host='host',port=port,user='user',password='password',db='schema') #creating cursor cursor= connection.cursor() #query for create Table TABLES={} TABLES['employees'] = ( "CREATE TABLE employees (" "PersonID int," "LastName varchar(255)," "FirstName varchar(255)," "Address varchar(255)," "City varchar(255)" ")") for name, ddl in TABLES.items(): cursor.execute(ddl) connection.commit() connection.close() Insert Data into the MySQL Table using Python import pymysql # Data set to insert insert_people=[("8","Perera","L.D","123",...

Basics of Data Science with Python

About Data Science with Python From this module I will post some interesting posts about Data science and how to gather information and process them using Python and for that I will be using below softwares  Python 3.6 Anaconda Jupyter Notebook PyCharm by JetBrains In the posts I will include fundamentals of Python  programming techniques such as lambdas, reading and manipulating csv files, and the numpy library. And In the posts I will be doing a simple Data Challange which is available on Kaggle and I will provide the URL's to the github repo which I will be posting the whole module once it finished. In Future I will be post some articles related to Applied Data Science with Python: Applied Plotting, Charting & Data Representation in Python, Applied Machine Learning in Python, Applied Text Mining in Python, Applied Social Network Analysis in Python

Background Processes - Oracle Architecture

SMON - System Monitor  Performs recovery after instance failure, monitor temporary segments and extents clean those temporary extents and segments.  PMON - Process Monitor Failed process resources in shared server architecture monitor and restart any failed dispatcher or server process.  DBWn - Database Writer Database Writer process writes new or changed data blocks into the data-files which are in the buffer cache.  LGWR - Log Writer Log writer is in change of REDO Log buffer management log writer is one of most active processes until log writer success data commit will not be completed. ARCn - Archive Process Copies REDO Logs to one or more destinations. CKPT - Checkpoint Process This helps to reduce the amount of time required for instance recovery. During checkpoint updates the header of the control file and data-files to reflect the last successful  System Changed Number (SCN)  This automatically starts every time the ...

Oracle Memory Structure - Oracle Architecture

SGA - System Global Area SGA is a  group of shared memory area which is dedicated to oracle instance. Shared Pool Shared pool is dividing into two major areas which are : Library Cache :  Area which stores the SQL information such as select statements, cursors and execution plans. Data Dictionary Cache :  Which contains cache, user information and privileges, segments and extent information as well. Buffer Cache Buffer cache holds blocks of data from disk that have been recently read to satisfy a select statement or that contains modified data-blocks.  Redo Log Buffer Redo Log buffer holds the most recent changes to the data blocks in the data-files.    Large Pool This is an operational area of the SGA it is used for transactions that interact with more than one database. Java Pool Java Pool is used by the Oracle JVM ( Java Virtual Machine ) for all Java code & data within user session. Stream Pool Stream Pool is...

Oracle Data Dump Import and Export

Login to server : sqlplus / as sysdba exp SYSTEM/password FULL=y FILE=dba.dmp LOG=dba.log CONSISTENT=y or exp SYSTEM/password PARFILE=param.dat where param.dat contains the following information  FILE=dba.dmp   GRANTS=y   FULL=y   ROWS=y   LOG=dba.log To dump a single schema to disk. exp / FILE=dump.dmp OWNER=production Export Specific tables to disk exp SYSTEM/password FILE=export.dmp TABLES=(user1.table,user2.table) Export Data in one user  exp / FILE=export.dmp TABLES=(table1,table2) Using imp:   To import the full database exported in the example above. imp SYSTEM/password FULL=y FIlE=dba.dmp To import just the dept and emp tables from the scott schema imp SYSTEM/password FIlE=dba.dmp FROMUSER=scott TABLES=(dept,emp) To import tables and change the owner  imp SYSTEM/password FROMUSER=blake TOUSER=scott FILE=blake.dmp TABLES=(unit,manager) To import just the scott schema exported in t...

Insert script with multiple cursors and condition check

DECLARE CURSOR C1 IS   SELECT ID FROM TABLE_NAME_1 WHERE COLUMN IN ('');   CURSOR C2     IS       SELECT ID FROM TABLE_NAME_2 WHERE COLUMN IN ('');              CURSOR C3 (CP_TABLE_01_ID NUMBER,CP_TABLE_02_ID NUMBER)         IS           SELECT COUNT(*) AS COUNT_UP           FROM TABLE_NAME_3           WHERE COLUMN_CONDITION_01=CP_TABLE_02_ID           AND COLUMN_CONDITION_02=CP_TABLE_01_ID; COUNT_UP NUMBER; BEGIN FOR R1 IN C1 LOOP     FOR R2 IN C2     LOOP          OPEN C3(R1.ID,R2.ID);        FETCH C3 INTO COUNT_UP;        CLOSE C3;               IF (COUNT_UP=0) THEN           INSERT           INTO TABL...

Split String which has space and replace spaces with "/"

Sample String "ABCD123 ETS, NM" UPDATE TABLE_NAME SET COLUMN_NAME = SUBSTR( 'ABCD123 ETS, NM' , 0 , 4 ) || REPLACE ((SUBSTR(( REPLACE ( 'ABCD123 ETS, NM' , ', ' , '/' )), 5 ,INSTR(( REPLACE ( 'ABCD123 ETS, NM' , ', ' , ',/' )), ',' , 1 ) - 4 )), ' ' , '/' ) || REPLACE (SUBSTR( 'ABCD123 ETS, NM' ,INSTR( 'ABCD123 ETS, NM' , ',' , 1 ) + 2 ), 'NM' , 'NEW MEXICO' ) WHERE CONDITIONS;