Saturday 10 May 2014

LOG MINING IN ORACLE DATABASE


Suppose data in a table has been deleted. Audit trail was not enabled, therefore we do not have a clue of the user who has performed deletion and there are around 30 users that have DELETE rights on that table. Management want to know the name of the user, here we can use Oracle LOG MINING feature to filter out the transactions performed during that period.

Note: Make sure the Archive log must be enabled previously, if you want to use LOG Mining technique.

CONSIDERATIONS
Database : ORCL
Schema : SCOTT Table : STUDENTS
Date of Deletion : 10-MAY-2014

1. Gather Archived Log 
Make archived logs of 10-MAY-2014 available in your local hard disk. In my case it is already available at the Flash Recovery Area.

2 . Create Tablespace for Log Miner 
By default LOG MINER tables resides in SYSAUX tablespace, but it is efficient to keep it in a separate tablespace for easy manageability & maintenance.

SQLPLUS SYS@ORCL AS SYSDBA

CREATE TABLESPACE TS_LOGMNR DATAFILE ‘C:\ORACLE\PRODUCT\10.2.0\ORADATA\ORCL\TS_LOGMNR_01.DBF’ SIZE 25M AUTOEXTEND ON MAXSIZE UNLIMITED;

Use the following statement to move all of LOG MINER objects from SYSAUX to TS_LOGMNR tablespace.
 EXEC DBMS_LOGMNR_D.SET_TABLESPACE(TS_LOGMNR);

3) Start Log Mining Load Archive log file into Log Miner.
BEGIN DBMS_LOGMNR.ADD_LOGFILE(C:\ORACLE\PRODUCT\10.2.0\FLASH_RECOVERY_AREA\ORCLARCHIVELOG\2014_05_12O1_MF_1_2_7JY3ZNWV_.ARC, DBMS_LOGMNR.NEW);
 END;
/

Perform Log Mining 

BEGIN DBMS_LOGMNR.START_LOGMNR(OPTIONS=>DBMS_LOGMNR.DICT_FROM_ONLINE_CATALOG+ DBMS_LOGMNR.COMMITTED_DATA_ONLY);
 END;

Format query results.
SET TRIM ON SET LINES 200
COL USERNAME FORMAT A15
COL TIMESTAMP FORMAT A20
COL SESSION_INFO FORMAT A200

SPOOL C:LOGMNR.TXT
SELECT USERNAME, TO_CHAR(TIMESTAMP,’DD-MON-YYYYY HH24:MI:SS’) TIMESTAMP, SESSION_INFO, SQL_REDO, SQL_UNDO FROM V$LOGMNR_CONTENTS WHERE SEG_OWNER='SCOTT’ AND SEG_NAME=’STUDENTS’ AND OPERATION=’DELETE’;
SPOOL OFF;

Stop Log Miner.
BEGIN DBMS_LOGMNR.END_LOGMNR;
END;
/

Open C:LOGMNR.TXT and look for DELETE statements. If you do not locate the problematic query,repeat the step 2 (above) with another archive log file, and keep repeating it until you caught the query and theuser that has performed the DELETE operation.

Note: V$LOGMNR_CONTENTS has many columns, and you can customize your results by modifying the abovequery.

Sunday 30 March 2014

Rapid Growth of SYSAUX Tablespace


Oracle 10g onwards, The SYSAUX an auxiliary tablespace to the SYSTEM tablespace created in your database.
Some database components that formerly created and used separate tablespaces now occupy the SYSAUX tablespace.
Below table shows the components that uses the SYSAUX tablespace as their default tablespace and the tablespace in which they were stored in earlier releases.

Component Using SYSAUX
Tablespace in Earlier Releases
Analytical Workspace Object Table
SYSTEM
Enterprise Manager Repository
OEM_REPOSITORY
LogMiner
SYSTEM
Logical Standby
SYSTEM
OLAP API History Tables
CWMLITE
Oracle Data Mining
ODM
Oracle Spatial
SYSTEM
Oracle Streams
SYSTEM
Oracle Text
DRSYS
Oracle Ultra Search
DRSYS
Oracle interMedia ORDPLUGINS Components
SYSTEM
Oracle interMedia ORDSYS Components
SYSTEM
Oracle interMedia SI_INFORMTN_SCHEMA Components
SYSTEM
Server Manageability Components

Statspack Repository
User-defined
Oracle Scheduler

Workspace Manager
SYSTEM

The statistics are stored in the data dictionary. Now whenever statistics in the dictionary are modified, old versions of statistics are saved automatically for future restoring. This history information is stored in SYSAUX tablespace and tables involved in this are as below:
WRI$_OPTSTAT_OPR                    
WRI$_OPTSTAT_AUX_HISTORY                
WRI$_OPTSTAT_TAB_HISTORY                
WRI$_OPTSTAT_IND_HISTORY                
WRI$_OPTSTAT_HISTGRM_HISTORY            
WRI$_OPTSTAT_HISTHEAD_HISTORY         
 
By default, the MMON performs the automatic purge that removes all stats history older than the following:
* current time - statistics history retention (by default 31 days)
* time of recent analyze in the system - 1

MMON performs the purge of the optimizer stats history automatically, but it has an internal limit of 5 minutes to perform this job.MMON will do this activity once in 24 hrs. If the operation takes more than 5 minutes, then it is aborted and stats not purged. No trace or alert message is reported. Because of this, as time elapse more data will be accommodated in above tables.
Statistics history retention is configurable using the ALTER_STATS_HISTORY_RETENTION procedure.


I have an Oracle 11g R2 database where I noticed the SYSAUX tablespace was growing larger every day.   After running $ORACLE_HOME/rdbms/admin/awrinfo.sql (Doc ID 1292724.1),
I found the largest consumer to be SM/OPTSTAT(9.5GB) and SM/AWR(1GB) as shown below.

  (1b) SYSAUX occupants space usage (v$sysaux_occupants)
********************************************************
|
| Occupant Name    Schema Name     Space Usage
| -------------------- -------------------- ----------------
| SM/OPTSTAT           SYS                        9,415.1 MB
| SM/AWR                    SYS                        1,005.3 MB
| SM/ADVISOR           SYS                        188.5 MB
| XDB                             XDB                       125.8 MB
| EM                              SYSMAN                 82.3 MB


(3b) Space usage within AWR Components (> 500K)
**********************************
COMP        MB     SEGMENT_NAME - % SPACE_USED                                                                                            
SEGMENT_TYPE
--------- --------- ---------------------------------------------------------------------
------------------------------  ---------------
ASH           382.0  WRH$_ACTIVE_SESSION_HISTORY.WRH$_ACTIVE_3563904870_4350 -  97%  TABLE PARTITION
ASH            38.0  WRH$_ACTIVE_SESSION_HISTORY_PK.WRH$_ACTIVE_3563904870_4350 - 98%  INDEX PARTITION

(4) Space usage by non-AWR components (> 500K)
**********************************

COMPONENT        MB SEGMENT_NAME                                                          SEGMENT_TYPE
--------- --------- --------------------------------------------------------------------- ---------------
NON_AWR     3,220.0 SYS.I_WRI$_OPTSTAT_HH_OBJ_ICOL_ST                    INDEX
NON_AWR     2,905.0 SYS.WRI$_OPTSTAT_HISTHEAD_HISTORY                TABLE
NON_AWR     1,930.0 SYS.I_WRI$_OPTSTAT_HH_ST                                          INDEX
NON_AWR       448.0 SYS.I_WRI$_OPTSTAT_H_OBJ#_ICOL#_ST                   INDEX
NON_AWR       296.0 SYS.WRI$_OPTSTAT_HISTGRM_HISTORY                    TABLE
NON_AWR       232.0 SYS.I_WRI$_OPTSTAT_H_ST                                               INDEX
NON_AWR       168.0 SYS.I_WRI$_OPTSTAT_TAB_OBJ#_ST                             INDEX
NON_AWR       141.0 SYS.SYS_LOB0000006306C00038$$                               LOBSEGMENT
NON_AWR       104.0 SYS.I_WRI$_OPTSTAT_TAB_ST                                          INDEX


To resolve this:

1. Turn off the Autoextend on the SYSAUX at the earliest to ensure that the tablespace doesn’t grow out of bounds and finally become complete unmanageable.

2. Manually purge old statistics using DBMS_STATS.purge_stats as below

Find out your present retention value using the below statement
SQL> select dbms_stats.get_stats_history_retention from dual;

GET_STATS_HISTORY_RETENTION
---------------------------
                         31

Find out the oldest statistics history using below statement(Shows available stats that have not been purged):
 

SQL> select dbms_stats.get_stats_history_availability from dual;

GET_STATS_HISTORY_AVAILABILITY
---------------------------------------------------------------------------
05-MAR-14 05.31.04.053232000 AM +05:30

Set retention of old stats to less number of days. I set here it to 10 days as below.

SQL> exec dbms_stats.alter_stats_history_retention(10);

PL/SQL procedure successfully completed.

SQL> select dbms_stats.get_stats_history_retention from dual;

GET_STATS_HISTORY_RETENTION
---------------------------
                         10
Purge stats older than 10 days. Best to do this in stages if there is a lot of data (sysdate-10) since it consumes more resources. Do this during less activity on the database. This purge will delete data from WRI$ tables.

Below command will purge stats which are older than 10 days.

SQL> exec dbms_stats.purge_stats(sysdate-10);

PL/SQL procedure successfully completed.

Below command shows available stats that have not been purged

SQL> select dbms_stats.get_stats_history_availability from dual;

GET_STATS_HISTORY_AVAILABILITY
---------------------------------------------------------------------------
30-MAR-14 09.46.46.000000000 AM +05:30

Once purge is done, reorg these tables to release space to the database. Refer ID 1271178.1 for more details.


Saturday 22 February 2014

Impdp does not created user/schema

Until  now my understanding for EXPDP/IMPDP was that if the user performing import has 'IMPORT FULL DATABASE' privilege, it'll create the users/schemas in the target database. I needed to export (expdp) schema from one machine and import (impdp) it to another machine. I just wanted  impdp to create schema on target database.

While doing  import  i encountered error
$ impdp system/oracle123 directory=TEST_DIR dumpfile=test.dmp logfile=impdptest.log
 .......
Starting "SYSTEM"."SYS_IMPORT_FULL_01":  system/******** directory=TEST_DIR dumpfile=USR_TEST.dmp logfile=impdpUSR_TEST.log
Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA
ORA-39083: Object type PROCACT_SCHEMA failed to create with error:
ORA-31625: Schema USR_TEST is needed to import this object, but is unaccessible
ORA-01435: user does not exist
Failing sql is:
BEGIN
sys.dbms_logrep_imp.instantiate_schema(schema_name=>SYS_CONTEXT('USERENV','CURRENT_SCHEMA'), export_db_name=>'XXX.YYY.COM', inst_scn=>'7788478540892');COMMIT; END;
Processing object type SCHEMA_EXPORT/TABLE/TABLE
ORA-39083: Object type TABLE:"USR_TEST"."TABLE1" failed to create with error:
ORA-01918: user 'USR_TEST' does not exist
Failing sql is:
CREATE TABLE "USR_TEST"."TABLE1" ("COLUMN1" VARCHAR2(20 BYTE) NOT NULL ENABLE, "COLUMN2" VARCHAR2(20 BYTE), "COLUMN3" VARCHAR2(20 BYTE)) SEGMENT CREATION IMMEDIATE PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 NOCOMPRESS LOGGING STORAGE(INITIAL 65536 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645 PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT FLASH_CACHE DEFAU
Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX
Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT
Job "SYSTEM"."SYS_IMPORT_FULL_01" completed with 2 error(s).
$

After reading some documents,blogs I discovered  that If  the schema you are importing to does not already exist, the import operation creates it, provided the dump file set contains the necessary CREATE USER metadata and you are importing with enough privileges.

Meaning the oracle user that exported the schema, should have had CREATE USER privilege.Export was done with user (USR_TEST) was a basic user and not DBA/create-user-privileged. 
Below query given result as.
SQL>  select  from  ROLE_SYS_PRIVS  where  PRIVILEGE = ’CREATE USER’ ;
ROLE                                              PRIVILEGE                     ADM
----------------------------           --------------------              ---
DBA                                               CREATE USER                 YES
IMP_FULL_DATABASE           CREATE USER                  NO
2 rows selected.
It means EXP_FULL_DATABAS role does not have create user privilege.

I did 'GRANT CREATE USER TO USR_TEST' here and  ran the export again and this time some additional "object types" were processed.
 when USR_TEST user didn't have 'CREATE USER' privilege:
 $ expdp USR_TEST/USR_TEST schemas=USR_TEST content=METADATA_ONLY directory=TEST_DIR dumpfile=USR_TEST.dmp logfile=expdpUSR_TEST.log
.......
Starting "USR_TEST"."SYS_EXPORT_SCHEMA_01":  USR_TEST/******** schemas=USR_TEST content=METADATA_ONLY directory=TEST_DIR dumpfile=USR_TEST.dmp logfile=expdpUSR_TEST.log
Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA
Processing object type SCHEMA_EXPORT/TABLE/TABLE
Processing object type SCHEMA_EXPORT/TABLE/COMMENT
Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX
Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT
Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Master table "USR_TEST"."SYS_EXPORT_SCHEMA_01" successfully loaded/unloaded
******************************************************************************
.......


when USR_TEST user  have 'CREATE USER' privilege:
 $ expdp USR_TEST/USR_TEST schemas=USR_TEST content=METADATA_ONLY directory=TEST_DIR dumpfile=USR_TEST.dmp logfile=expdpUSR_TEST.log
.......
Starting "USR_TEST"."SYS_EXPORT_SCHEMA_01":  USR_TEST/******** schemas=USR_TEST content=METADATA_ONLY directory=TEST_DIR dumpfile=USR_TEST.dmp logfile=expdpUSR_TEST.log
Processing object type SCHEMA_EXPORT/USER
Processing object type SCHEMA_EXPORT/SYSTEM_GRANT
Processing object type SCHEMA_EXPORT/ROLE_GRANT
Processing object type SCHEMA_EXPORT/DEFAULT_ROLE
Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA
Processing object type SCHEMA_EXPORT/TABLE/TABLE
Processing object type SCHEMA_EXPORT/TABLE/COMMENT
Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX
Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT
Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Master table "USR_TEST"."SYS_EXPORT_SCHEMA_01" successfully loaded/unloaded
******************************************************************************
.......

Note the Processing object type SCHEMA_EXPORT/USER, SCHEMA_EXPORT/SYSTEM_GRANT, SCHEMA_EXPORT/ROLE_GRANT, SCHEMA_EXPORT/DEFAULT_ROLE.

To avoid such situations always try to take the export using SYSTEM user.

Identify the Tablespace names from EXPDP dump file


 I got  request from development team to import some old data in UAT region.The expdp dump file received from backup team was without  log file and i have no access to the export file's source DB. I wanted to import that dumpfile into UAT DB and i do not know the tablespace names required to do so.

Solution:
Use the sqlfile option present in impdp utility. This option will write all the SQL DDL to a specified file instead of executing them.

$ impdp scott/tiger directory=exp_dir dumpfile=scott.dmp sqlfile=script.sql

Import: Release 11.1.0.7.0 - 64bit Production on Tuesday, 20 February, 2014 21:34:36

Copyright (c) 2003, 2007, Oracle.  All rights reserved.

Connected to: Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
Master table "SCOTT"."SYS_SQL_FILE_FULL_01" successfully loaded/unloaded
Starting "SCOTT"."SYS_SQL_FILE_FULL_01":  scott/******** directory=exp_dir dumpfile=scott.dmp sqlfile=script.sql
Processing object type SCHEMA_EXPORT/USER
Processing object type SCHEMA_EXPORT/SYSTEM_GRANT
Processing object type SCHEMA_EXPORT/ROLE_GRANT
Processing object type SCHEMA_EXPORT/DEFAULT_ROLE
Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA
Processing object type SCHEMA_EXPORT/SEQUENCE/SEQUENCE
Processing object type SCHEMA_EXPORT/TABLE/TABLE
Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX
Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT
Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type SCHEMA_EXPORT/PROCEDURE/PROCEDURE
Processing object type SCHEMA_EXPORT/PROCEDURE/ALTER_PROCEDURE
Processing object type SCHEMA_EXPORT/TABLE/TRIGGER
Processing object type SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
Job "SCOTT"."SYS_SQL_FILE_FULL_01" successfully completed at 21:34:53

The content of the scripts.sql file would be like this.
-- CONNECT SCOTT
ALTER SESSION SET EDITION = "ORA$BASE";
-- new object type path: SCHEMA_EXPORT/USER
-- CONNECT SYSTEM
ALTER SESSION SET EDITION = "ORA$BASE";
 CREATE USER "SCOTT" IDENTIFIED BY VALUES 'S:D846EA3EB87287A3AED08AF38EB0B4F640F49A9A4A972108BF3917B769;DB1B37F84BDF15E6'
      DEFAULT TABLESPACE "USERS"
      TEMPORARY TABLESPACE "TEMP";

-- new object type path: SCHEMA_EXPORT/SYSTEM_GRANT
GRANT UNLIMITED TABLESPACE TO "SCOTT";

-- new object type path: SCHEMA_EXPORT/ROLE_GRANT
 GRANT "DBA" TO "SCOTT";

-- new object type path: SCHEMA_EXPORT/DEFAULT_ROLE
 ALTER USER "SCOTT" DEFAULT ROLE ALL;

-- new object type path: SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA
-- CONNECT SCOTT
ALTER SESSION SET EDITION = "ORA$BASE";

BEGIN
sys.dbms_logrep_imp.instantiate_schema(schema_name=>SYS_CONTEXT('USERENV','CURRENT_SCHEMA'), export_db_name=>'PROD9.DIAMOND.COM', inst_scn=>
'11626845804212');
COMMIT;
END;
/
 .
.
.
.
.
.


After finding all the required tablespaces i used REMAP_TABLESPACE  and then import the dump file .
One can manually create same  tablespaces and import the dump file. 


Wednesday 29 January 2014

ORA-01031: insufficient privileges to sys@standby as sysdba


The ORA-01031 is very generic error and normally users will receive this error when users don’t have appropriate privilege.

$oerr ora 01031
01031, 00000, "insufficient privileges"
// *Cause: An attempt was made to change the current username or password
//         without the appropriate privilege. This error also occurs if
//         attempting to install a database without the necessary operating
//         system privileges.
//         When Trusted Oracle is configure in DBMS MAC, this error may occur
//         if the user was granted the necessary privilege at a higher label
//         than the current login.
// *Action: Ask the database administrator to perform the operation or grant
//          the required privileges.
//          For Trusted Oracle users getting this error although granted the
//          the appropriate privilege at a higher label, ask the database
//          administrator to regrant the privilege at the appropriate label.

On standby
$ sqlplus
Enter user-name: /as sysdba

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL>


From Primary or Standby
$ sqlplus
 

Enter user-name: sys@standby as sysdba
Enter password:
ERROR:
ORA-01031: insufficient privileges


Error in alert log of PRIMARY database

Error 1034 received logging on to the standby
PING[ARC0]: Heartbeat failed to connect to standby 'STANDBY'. Error is 1034.
Error 1034 received logging on to the standby
FAL[server, ARC2]: Error 1034 creating remote archivelog file 'STANDBY'

 Metalink notes for insufficient privileges  as sysdba
OERR: ORA 1031 "insufficient privileges" [ID 18622.1]
Troubleshooting ORA-1031: Insufficient Privileges While Connecting As SYSDBA [ID 730067.1]

But

Original problem in my case  was with listener file. Instance name is crated in small letter but in listener.ora sid given in CAPS it caused the problem. After changing sid in listner.ora it is working fine.

Total Pageviews