Thursday, October 24, 2013

11g DR build manually using rman backup

1.copy/scp Oracle home from primary and relink

2.Modify Init and tns files as below on standby

[oratpr@drhost  dbs]$


db_unique_name             = SID
*.FAL_CLIENT='SID'
*.fal_server='SID_STBY'
*.log_archive_config='dg_config=(SID,SID_STBY)'
*.LOG_ARCHIVE_DEST_1='LOCATION=USE_DB_RECOVERY_FILE_DEST MANDATORY'
*.LOG_ARCHIVE_DEST_2='service=SID_STBY valid_for=(online_logfiles,primary_role) db_unique_name=SID LGWR ASYNC=20480 OPTIONAL REOPEN=15 NET_TIMEOUT=30'
*.log_archive_dest_state_2='DEFER'
*.log_archive_dest_state_3='DEFER'
*.LOG_ARCHIVE_FORMAT='%t_%s_%r.dbf'
*.db_recovery_file_dest='/archivelogs/SID'
*.db_recovery_file_dest_size=100G




[oratpr@drhost SID_drhost]$ cat SID_drhost_ifile.ora
SID_STBY=
    (DESCRIPTION=
        (ADDRESS=(PROTOCOL=tcp)(HOST=primaryhost)(PORT=1571))
      (CONNECT_DATA=
        (SID=SID)
      )
    )


3.Modify init/tns  file on primary to include following


init
-----

*.DB_UNIQUE_NAME='SID'
*.FAL_CLIENT='SID'
*.fal_server='SID_STBY'
*.log_archive_config='dg_config=(SID,SID_STBY)'
*.LOG_ARCHIVE_DEST_1='LOCATION=USE_DB_RECOVERY_FILE_DEST MANDATORY'
*.LOG_ARCHIVE_DEST_2='service=SID_STBY LGWR ASYNC=20480 valid_for=(online_logfiles,primary_role) db_unique_name=SID_STBY OPTIONAL REOPEN=15 NET_TIMEOUT=30'
*.log_archive_dest_state_2='DEFER'
*.log_archive_dest_state_3='DEFER'
*.LOG_ARCHIVE_FORMAT='%t_%s_%r.dbf'
*.standby_archive_dest='LOCATION=USE_DB_RECOVERY_FILE_DEST'
*.standby_file_management='AUTO'
*.remote_login_passwordfile='EXCLUSIVE'


tns
---
[oratpr@primaryhost SID_primaryhost]$ cat SID_primaryhost_ifile.ora

SID_STBY = (DESCRIPTION=
                (ADDRESS=(PROTOCOL=tcp)(HOST=drhost.cm.charter.com)(PORT=1571))
                (CONNECT_DATA=(SID=SID_stdby))
            )






4. Mount /copy backup on DR and start restore as below

a. startup database with nomount

b.Validate that you able to connect sys/***@sid_primary as sysdba
If not scp password file from primary to standby

c. Connect to rman and run restore


rman target sys/****@sid_stdby auxiliary sys/****@sid_prim
sql 'alter session set optimizer_mode=rule';
DUPLICATE TARGET DATABASE FOR STANDBY DORECOVER NOFILENAMECHECK;



d.  After restore is done it will complain for archives
If required , restore missing archives from backup .
on standby
rman target /
restore archivelog from sequence <.....missing sequenece fro which restore failed >;



5. Enable archive shipping  on primary , restart db if required

*.log_archive_dest_state_2='enable'

6.Initiate redo apply on standby

alter database recover managed standby database disconnect;


7. Check alert log and validate that db/primary are in sync






For opening DR

==================



SQL> RECOVER MANAGED STANDBY DATABASE CANCEL;
Media recovery complete.
SQL> ALTER DATABASE ACTIVATE STANDBY DATABASE;

Database altered.

SQL> shut immediate
ORA-01109: database not open


Database dismounted.
ORACLE instance shut down.
SQL> startup
ORACLE instance started.

Total System Global Area 1073741824 bytes
Fixed Size                  2089336 bytes
Variable Size             427822728 bytes
Database Buffers          629145600 bytes
Redo Buffers               14684160 bytes
Database mounted.
Database opened.

10g/11g database creation manually

set time on timing on feed on
spool cr_db.log
CREATE DATABASE "RMANDR"
MAXINSTANCES 8
MAXLOGHISTORY 1
MAXLOGFILES 16
MAXLOGMEMBERS 3
MAXDATAFILES 1024
CHARACTER SET UTF8
NATIONAL CHARACTER SET AL16UTF16
DATAFILE '/oracle/RMAN/oradata/system1a.dbf' SIZE 1000M extent management local
SYSAUX DATAFILE '/oracle/RMAN/oradata/sysaux1a.dbf' SIZE 1000M
DEFAULT TEMPORARY TABLESPACE TEMP TEMPFILE '/oracle/RMAN/oradata/temp1a.dbf' SIZE 1000M
extent management local uniform size 1m
UNDO TABLESPACE "UNDOTBS1" DATAFILE '/oracle/RMAN/oradata/undotbs1a.dbf' SIZE 1000M
LOGFILE
GROUP 1 ('/oracle/RMAN/oradata/log1a.dbf','/oracle/RMAN/oradata/log1b.dbf') SIZE 200M,
GROUP 2 ('/oracle/RMAN/oradata/log2a.dbf','/oracle/RMAN/oradata/log2b.dbf') SIZE 200M,
GROUP 3 ('/oracle/RMAN/oradata/log3a.dbf','/oracle/RMAN/oradata/log3b.dbf') SIZE 200M
USER SYS IDENTIFIED BY manager
USER SYSTEM IDENTIFIED BY manager
/



connect /as sysdba
spool dictionary.out
@?/rdbms/admin/catalog.sql
@?/rdbms/admin/catblock.sql
@?/rdbms/admin/catproc.sql
@?/rdbms/admin/catoctk.sql
@?/rdbms/admin/owminst.plb
@?/javavm/install/initjvm.sql
@?/xdk/admin/initxml.sql
@?/xdk/admin/xmlja.sql
@?/rdbms/admin/catjava.sql
@?/rdbms/admin/catexf.sql
@?/rdbms/admin/catclust.sql

connect SYSTEM/manager
spool sqlplus.log
@?/sqlplus/admin/pupbld.sql
@?/sqlplus/admin/help/hlpbld.sql helpus.sql
spool off

spool off

10g/11g DB tablespace free query

col "Tablespace" for a22
col "Used MB" for 99,999,999
col "Free MB" for 99,999,999
col "Total MB" for 99,999,999

select df.tablespace_name "Tablespace",
totalusedspace "Used MB",
(df.totalspace - tu.totalusedspace) "Free MB",
df.totalspace "Total MB",
round(100 * ( (df.totalspace - tu.totalusedspace)/ df.totalspace))
"Pct. Free"
from
(select tablespace_name,
round(sum(bytes) / 1048576) TotalSpace
from dba_data_files
group by tablespace_name) df,
(select round(sum(bytes)/(1024*1024)) totalusedspace, tablespace_name
from dba_segments
group by tablespace_name) tu
where df.tablespace_name = tu.tablespace_name ;

Wednesday, October 23, 2013

Workflow mailer logs shwoing SSL handshake failed: X509CertChainIncompleteErr

This issue is due to java store certs are not proper .

Create you cacerts as below .
Also make sure that cacerts path is properly defined in the mailer .

Before running these steps Make sure that your CA.crt file is proper and it has intermediate certificate included .


1. Validate you mailer has correct cacerts locations

Execute $FND_TOP/sql/afsvcpup.sql

update Notification mailer truststore parameter to path of cacert


10243 SSL Trust store  /util/jdk1.6.0_21/jre/lib/security/cacerts



2. Create your java cacerts as below (default password will be changeit )


cd ssl.crt

mv $OA_JRE_TOP/jre/lib/security/cacerts $OA_JRE_TOP/jre/lib/security/cacerts.orig
$OA_JRE_TOP/jre/bin/keytool -import -alias ApacheServer -file server.crt -trustcacerts -v -keystore $OA_JRE_TOP/jre/lib/security/cacerts
$OA_JRE_TOP/jre/bin/keytool -import -alias ApacheCA -file ca.crt -trustcacerts -v -keystore $OA_JRE_TOP/jre/lib/security/cacerts


3. Bounce all services including CM .



Hope this resolves .

Grid 11gR2 cloning/ASM Cloning

1. copy of 11g GRID infrastructure from ealrier installed env .

a. Copy the above zipped file over to the target server.

b. Then untar the home.


c. If untar is done as root, make sure, to change the home ownership.



2. Set up Grid Infrastructure Home

a .

$ cd /clone/bin

$ perl ./clone.pl ORACLE_HOME= ORACLE_BASE=  OSDBA_GROUP=dba

The output of above command will be similar to below.


./runInstaller -clone -waitForCompletion  "ORACLE_HOME=/app/grid/11.2.0/grid" "ORACLE_BASE=/oracle/APP/grid" "oracle_install_OSDBA=dba" -defaultHomeName -silent -noConfig -nowait
Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 16383 MB    Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2013-05-29_11-29-22PM. Please wait ...Oracle Universal Installer, Version 11.2.0.3.0 Production
Copyright (C) 1999, 2011, Oracle. All rights reserved.

You can find the log of this install session at:
 /home/oragrid/oraInventory/logs/cloneActions2013-05-29_11-29-22PM.log
.................................................................................................... 100% Done.



Installation in progress (Wednesday, May 29, 2013 11:29:34 PM CDT)
........................................................................                                                        72% Done.
Install successful

Linking in progress (Wednesday, May 29, 2013 11:29:38 PM CDT)
Link successful

Setup in progress (Wednesday, May 29, 2013 11:30:04 PM CDT)
.................                                               100% Done.
Setup successful

End of install phases.(Wednesday, May 29, 2013 11:30:25 PM CDT)
WARNING:A new inventory has been created in this session. However, it has not yet been registered as the central inventory of this system.
To register the new inventory please run the script '/home/oragrid/oraInventory/orainstRoot.sh' with root privileges.
If you do not register the inventory, you may not be able to update or patch the products you installed.
The following configuration scripts need to be executed as the "root" user.
/home/oragrid/oraInventory/orainstRoot.sh
/oracle/APP/grid/11.2.0/grid/root.sh
To execute the configuration scripts:
    1. Open a terminal window
    2. Log in as "root"
    3. Run the scripts

Run the script on the local node.
The cloning of OraHome1 was successful.
Please check '/home/oragrid/oraInventory/logs/cloneActions2013-05-29_11-29-22PM.log' for more details.



As per above message, run the root.sh scripts as root as shown below.



3. Run root scripts

a. /home/grid/oraInventory/orainstRoot.sh
Changing permissions of /home/oragrid/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /home/oragrid/oraInventory to dba.
The execution of the script is complete.



b. /oracle/APP/grid/11.2.0/grid/root.sh
Check /oracle/APP/grid/11.2.0/grid/install/root_2013-05-29_23-34-18.log for the output of root script



4. Now register CRS as root.


c. /perl/bin/perl -I/perl/lib -I/crs/install /crs/install/roothas.pl

The output will be similar to below.

Using configuration parameter file: /oracle/APP/grid/11.2.0/grid/crs/install/crsconfig_params
User ignored Prerequisites during installation
LOCAL ADD MODE
Creating OCR keys for user 'oragrid', privgrp 'dba'..
Operation successful.
LOCAL ONLY MODE
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4664: Node abxyzz successfully pinned.
Adding Clusterware entries to inittab

abcxyz      2013/05/14 02:50:14     /oracle/APP/grid/11.2.0/grid/cdata/abcxyz/backup_20130514_025014.olr
Successfully configured Oracle Grid Infrastructure for a Standalone Server



5. Start CRS.


a. cd /bin

b. ./crsctl check has
CRS-4638: Oracle High Availability Services is online

c. ./crsctl check css
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
d. ./crsctl modify resource "ora.cssd" -attr "AUTO_START=1"
e. ./crsctl stop has

f. ./crsctl start has

g.  ./crsctl check has
h ./crsctl check css

ps -eaf |grep cssd ---this will show cssd.bin running

New changes coming in R12.2

New changes coming in R12.2 , this is with reference to R12.1


1.Oracle Fusion Middleware 11g R1 PS3 (11.1.1.4) including WebLogic 10.3.4 as its application server

2. Default Database is 11g R2 (11.2.X)

3. Oracle JSP Compiler (OJSP) 10.1.3.5 (12.0.X and 12.1.X) is replaced by WebLogic JSP Compiler 11.1.1.4

4.Online Patching (OLP) introduced in Oracle Apps 12.2

5. Oracle Apps R12.2 cloning will also support Fusion Middleware cloning

Sunday, October 20, 2013

ssl certificate conversion to oracle wallet

Source your env and use below tool and syntax to do this .Make sure you have all 3 files (certificate, key, root certificate available before starting .)

$ORACLE_HOME/Apache/Apache/bin/ssl2ossl -cert $ORACLE_HOME/Apache/Apache/conf/ssl.wlt/default/certs/server.crt -key $ORACLE_HOME Apache/Apache/conf/ssl.wlt/default/certs/server.key -cafile $ORACLE_HOME Apache/Apache/conf/ssl.wlt/default/certs/ca.crt -wallet $ORACLE_HOME/Apache/Apache/conf/ssl.wlt/default/cbeytest -ssowallet yes
Enter wallet password:
Verifying password - Enter wallet password:
SUCCESS



Doing bulk delete from oid

1)Take backup of OID

2)Disable sync profile

$ dipassistant mp -host -port 389 -passwd ***** -profile ActiveChgImp odip.profile.status=DISABLE
Profile successfully modified.

Note: For security reasons it is not recommended to provide a password on the command line, unless you're being prompted for it.

3)Stop DIP process

oidctl connect=oid server=ODISRV instance=2 configset=1 flags="host=oid01.oracle.net grpid=configset1 port=389 " stop
NLS_LANG not set in environment
Setting NLS_LANG to AMERICAN_AMERICA.AL32UTF8
oidctl:Waiting for oidmon to stop ODISRV (instance=2) pid=10157
oidctl:Waiting for oidmon to stop ODISRV (instance=2) pid=10157
oidctl:Stopped ODISRV (instance=2) successfully

4)Get list of all users to be deleted
ldapsearch -h oid01.oracle.net -p 389 -D "cn=orcladmin" -w **** -s sub -b "cn=Users,dc=com" "(objectclass=*)" dc >> \tmp\test.ldif

5)Delete following from the user list created in step 4, as we want to retain these contianers

vi \tmp\test.ldif
cn=Users, dc=com
cn=orcladmin, cn=Users, dc=com
cn=PUBLIC, cn=Users, dc=com

6)Do bulk delete



ldapdelete -h oid01.oracle.net -p 389 -D "cn=orcladmin" -w ***** -c -v -f \tmp\test.ldif

How to get orasso password from oid

ldapsearch -h -p -D "cn=orcladmin" -w "*****" -s sub "orclResourceName=orasso" orclpasswordattribute

ldapbind on 636 ssl port using oracle wallet

ldapbind -h -p -D -w -U 2 -W "file: " -P


ldapbind -h -p 636 -D "test_oracle" -w -U 2 -W "file:abc/zyx/wallet" -P

FND Queue rebuild

1. Stop all middle tier services including the concurrent managers. Make sure no FNDLIBR, FNDSM, or other dead processes are running.

2. Stop and then restart the database.


3. Rebuild Q

SQL> exec DBMS_AQADM.STOP_QUEUE('FND_CP_GSM_IPC_AQ');


PL/SQL procedure successfully completed.

SQL> SQL> exec DBMS_AQADM.DROP_QUEUE('FND_CP_GSM_IPC_AQ');


PL/SQL procedure successfully completed.

SQL> SQL> exec DBMS_AQADM.DROP_QUEUE_TABLE('FND_CP_GSM_IPC_AQTBL');


PL/SQL procedure successfully completed.

SQL> SQL> exec DBMS_AQADM.CREATE_QUEUE_TABLE(queue_table => 'FND_CP_GSM_IPC_AQTBL', -
> queue_payload_type => 'SYSTEM.FND_CP_GSM_IPQ_AQ_PAYLOAD', -
> sort_list => 'priority,enq_time', -
> multiple_consumers => TRUE, -
> comment => 'IPC for GSM Generic Cartridge', -
> compatible => '8.1');


PL/SQL procedure successfully completed.

SQL> SQL>
SQL> exec DBMS_AQADM.CREATE_QUEUE ('FND_CP_GSM_IPC_AQ', FND_CP_GSM_IPC_AQTBL');

ERROR:
ORA-01756: quoted string not properly terminated


SQL> SQL> exec DBMS_AQADM.CREATE_QUEUE ('FND_CP_GSM_IPC_AQ','FND_CP_GSM_IPC_AQTBL');

PL/SQL procedure successfully completed.

SQL> exec DBMS_AQADM.START_QUEUE ('FND_CP_GSM_IPC_AQ');


PL/SQL procedure successfully completed.

SQL> SQL> exit








4.  Log in as applmgr to $FND_TOP/patch/115/sql
Run the script: afdcm037.sql

5. Relink FNDSM and FNDLIBR executables as mentioned below:
$ adrelink.sh force=y link_debug=y "fnd FNDLIBR"
$ adrelink.sh force=y link_debug=y "fnd FNDSM"

6. Run cmclean.sql

7. Start up the Services and retest.

ORA-00439 feature not enabled: Real Application Clusters

This fixed my issue .


SQL> startup pfile='/tmp/pfile-rac'

ORA-00439: feature not enabled: Real Application Clusters
ORA-01078: failure in processing system parameters

Because => An Oracle Home disabled RAC.

So, we should relink library to enable RAC on this Oracle Home

$ cd $ORACLE_HOME/rdbms/lib

$ make -f ins_rdbms.mk rac_on

$ make -f ins_rdbms.mk ioracle

startup now worked

Opening port on linux

I used below to open 7001 port from root as application was not able to reach .

iptables -t filter -I INPUT -p tcp --dport 7001 -j ACCEPT
service iptables save
iptables: Saving firewall rules to /etc/sysconfig/iptables:[  OK  ]
iptables -t filter -nL

FND_LOBS_CTX rebuiding

SQL> drop index APPLSYS.FND_LOBS_CTX;


SQL> select
FILE_FORMAT,
count(*)
from
fnd_lobs
group by FILE_FORMAT;  2    3    4    5    6

FILE_FORMA   COUNT(*)
---------- ----------
binary           5736
text            39437
IGNORE            484

SQL> update fnd_lobs
set FILE_FORMAT = 'IGNORE'
where NVL(PROGRAM_NAME,'NULL') != 'FND_HELP' ;  2    3

484 rows updated.

SQL> commit;

Commit complete.

SQL> select status
from all_indexes
where owner='APPLSYS'
and index_name='FND_LOBS_CTX'
and table_name='FND_LOBS';  2    3    4    5

no rows selected

SQL> @$FND_TOP/sql/aflobbld.sql applsys apps

PL/SQL procedure successfully completed.


Commit complete.

SQL> update fnd_lobs
set file_format = fnd_gfm.set_file_format(file_content_type);  2

45657 rows updated.

SQL> commit;

Commit complete.

SQL> select
FILE_FORMAT,
count(*)
from
fnd_lobs
group by FILE_FORMAT;  2    3    4    5    6

FILE_FORMA   COUNT(*)
---------- ----------
IGNORE           5736
TEXT            39879
BINARY             42

SQL> select status
from all_indexes
where owner='APPLSYS'
and index_name='FND_LOBS_CTX'
and table_name='FND_LOBS';  2    3    4    5

STATUS
--------
VALID

checking exalogic version

[root@abcdeee~]# rpm -qi exalogic.tools
Name        : exalogic.tools               Relocations: (not relocatable)
Version     : 2.0.0.0.0                         Vendor: Oracle
Release     : 1.0                           Build Date: Wed 21 Dec 2011 03:15:31 PM EST
Install Date: Wed 23 May 2012 12:31:43 AM EDT      Build Host: sdp-scream3.us.oracle.com
Group       : exalogic                      Source RPM: exalogic.tools-2.0.0.0.0-1.0.src.rpm
Size        : 242026                           License: GPL
Signature   : (none)
Packager    : Ravi Akella
URL         : http://www.oracle.com
Summary     : exalogic.tools
Description :
Exalogic tools package that provides some diagnostic tools

OID/SSO/Portal 10g ip/hostname change

Below is link we can use .
http://docs.oracle.com/cd/B14099_19/core.1012/b13995/host.htm#CHDGHFGE

Out of this below are applicable to us

8.2.2 Changing the Hostname, Domain Name, or IP Address of a Middle-Tier Installation

8.2.3 Changing the Hostname, Domain Name, or IP Address of an Identity Management Installation


Below are the commands to be run after move .Old ip needs to be noted before shutdown .

cd ORACLE_HOME/chgip/scripts
./chgiphost.sh -idm

cd ORACLE_HOME/chgip/scripts
./chgiphost.sh -mid