Oracle TFA And AHF log collection / Installation

 

Trace File Analyzer Collector also known as TFA is a diagnostic collection utility which greatly simplifies the diagnostic data collection for both Oracle Database as well as Oracle Clusterware/Grid Infrastructure RAC environments.

Trace File Analyzer provides a central and single interface for all diagnostic data collection and analysis.

When a problem occurs, TFA collects all the relevant data at the time of the problem and consolidates data even across multiple nodes in a clustered Oracle RAC environment.  Only the relevant diagnostic data is collected and can be packaged and uploaded to Oracle Support and this leads to faster resolution times. All the required diagnostic data is collected via a single tfactl command instead of having to individually look for the required diagnostic information across a number of database and cluster alert logs, trace files or dump files.

In addition to the core functionality of gathering, consolidating and processing diagnostic data, Trace File Analyzer comes bundled with a number of support tools which enable us to obtain a lot of other useful information like upgrade readiness, health checks for both Engineered as well as non-Engineered systems, OS performance graphs, Top SQL queries etc.

Oracle Trace File Analyzer is shipped along with Oracle Grid Infrastructure (from version 11.2.0.4). However, it is recommended to download the latest TFA version which can be accessed via the My Oracle Support Note 1513912.1 since the TFA bundled with the Oracle Grid Infrastructure does not include many of the new features, bug fixes and more importantly the Oracle Database Support Tools bundle.

Oracle releases new versions of the TFA several times a year and the most current version is Trace File Analyzer 18.1.1 which is now available for download via the MOS Note 1513912.1.

##################################################################################
TFA
##################################################################################

__________________
Install   Tfa 
__________________

Download the latest version of Oracle Trace File Analyzer with Oracle Database support tools bundle from My Oracle Support note 1513912.1

Upgrading is similar to first-time install. As root, use the installTFAplatform script example for linux > 

./installTFA-Linux.

For  Rac :  
Install Tfa on both Nodes  -->  Sync  on both Nodes  --> Start 

( As root ) 
./installTFAplatform 

( As Oracle User ) 
$ ./installTFAplatform -extractto dir -javahome jre_home

cd /u01/software
unzip TFA-LINUX_v18.2.1.zip
mkdir -p $ORACLE_HOME/tfa
./installTFA-LINUX -local -tfabase $ORACLE_HOME/tfa



After having completed this process on all nodes, let’s synchronize them. As I decided not to use SSH, I need to execute those final steps :

Launch tfactl syncnodes using sudo : 
# sudo /grid/infrastructure/home/bin/tfactl syncnodes       ( # $GIHOME/tfa/nodename/tfa_home/bin/synctfanodes.sh ) 

Login using root is disabled in sshd config. Please enable it or
Please copy these files manually to remote node and restart TFA

1. /grid/infrastructure/home/tfa/node01/tfa_home/server.jks
2. /grid/infrastructure/home/tfa/node01/tfa_home/client.jks
3. /grid/infrastructure/home/tfa/node01/tfa_home/internal/ssl.properties

These files must be owned by root and should have 600 permissions.


sudo /grid/infrastructure/home/bin/tfactl stop
sudo /grid/infrastructure/home/bin/tfactl start

__________________
Log Collection   Tfa 
__________________

 cd $ORACLE_HOME/tfa/bin/

# Gather diagnostic information about TFA itself.
./tfactl diagnosetfa -local

# Gather information about errors. You are prompted to select a specific incident.
./tfactl diagcollect -srdc ORA-00600
./tfactl diagcollect -srdc ORA-07445

# Collect data for all components for a specific time period.
./tfactl diagcollect -from "MON/DD/YYYY 13:00:00" -to  "MON/DD/YYYY 13:00:00"

or 

./tfactl diagcollect -all -from "MON/DD/YYYY 13:00:00" -to  "MON/DD/YYYY 13:00:00"  -noclassify 

# Collect data for all components for the last 12 hours.
./tfactl diagcollect

Trace File Analyzer Command Examples

• Viewing System and Cluster Summary

tfactl summary

• To find all errors in the last one day

tfactl analyze -last 1d

• To find all occurrences of a specific error  (in this case ORA-00600 errors)

tfactl analyze -search “ora-00600” -last 8h

• To set the notification email to use

tfactl set notificationAddress=joeblogs@oracle.com

• Enable or disable Automatic collections (ON by default)

tfactl set autodiagcollect=OFF

• Adjusting the Diagnostic Data Collection Period

tfactl diagcollect -last 1 h

tfactl diagcollect -from “2018-03-21″

tfactl diagcollect  from “2018-03-21” -to “2018-03-22”

• Analyze, trim and zip all files updated in the last 12 hours, including Cluster Health Monitor and OSWatcher data, from across all nodes  the cluster

tfactl diagcollect -all -last 12h

• Run collection from specific nodes in a RAC cluster

tfactl diagcollect -last 1d -node rac01

• Run collection for a specific database

tfactl -diagcollect -database hrdb -last 1d

• Uploading collections to Oracle Support

Execute tfactl setupmos to configure Oracle Trace File Analyzer with MOS user name and password followed by

tfactl diagcollect -last 1d -sr 1234567

• Search  database alert logs for the string “ORA-” from the past one day

tfactl analyze -search “ORA” -comp db -last 1d

• Display a summary of events collected from all alert logs and system logs from the past six hours

tfactl analyze -last 6h

• View the summary of a TFA deployment. This will display cluster node information as well as information related to database and grid infrastructure software homes like version, patches installed, databases running etc.

tfactl summary

• Grant access to a user

tfactl access add -user oracle

• List users with TFA access

tfactl access lsusers

• Run orachk

tfactl run orachk

• Display current configuration settings

tfactl print config

Commonly Used Commands : 

tfactl set reposizeMB=50240 

tfactl print repository 

tfactl purge -older 2h

tfactl set repositorydir=/u02/repository/tfa/ 

tfactl diagcollect -srdc ora600

tfactl diagcollect -srdc ora04030 

tfactl analyze -search "/ORA- /c" -comp db -last 2d 

tfactl diagcollect -all -from "DEC/14/2021 01:00:00 -to DEC/15/2021 03:00:00" 

tfactl diagcollect  -from "DEC/14/2021 01:00:00 -to DEC/15/2021 03:00:00"  

tfactl analyze -search "ORA-04031" -last 1d

tfactl analyze -since 1d

tfactl analyze -comp os -for ”Oct/01/2020 11" -search "."

tfactl analyze -comp osw -since 6h

tfactl analyze -search "ORA-" -since 2d

tfactl analyze -comp oswslabinfo -from ”Oct/01/2020 05:00:01" -to ”Oct/01/2020 06:00:01"

tfactl diagcollect –srdc dbperf

tfactl diagcollect -srdc ORA-00600

tfactl managelogs -show usage 

tfactl managelogs -purge -older 30d

tfactl tail alert

tfactl summary

-- Genearate complete summary overview in html

tfactl summary -html

-- Generate patching summary:

tfactl summary -patch -html

-- Generate asm summary

tfactl summary -asm -html

__________________
TFA  Status  : 
__________________

[root@ermantest tmp]# /u01/app/oracle/product/12.1.0.2/db_1/bin/tfactl status
root@ermantest tmp]# /u01/app/oracle/product/12.1.0.2/db_1/bin/tfactl toolstatus
root@ermantest tmp]# /u01/app/oracle/product/12.1.0.2/db_1/bin/tfactl version
# sudo /grid/infrastructure/home/bin/tfactl print config
# sudo /grid/infrastructure/home/bin/tfactl syncnodes



____________________________________
TFA installed as part of 21523375 (Oct2015 CPU)
____________________________________

Ater The Grid  Psu   TFA was installed.
If you need to shutdown processes running from grid, TFA will need to be stopped as well (# /etc/sinit.d/init.tfa stop) since crsctl stop crs does not stop TFA


__________________
Controlling TFA  cpu  usage 
__________________

tfactl setresourcelimit 
 [-tool tool_name] 
 [-resource resource_type] 
 [-value value]
To limit TFA to a maximum of 50% of a single CPU, run the following:
# tfactl setresourcelimit -value 0.5




__________________
TFA Reference : 
__________________

https://xy2401-local-doc-java.github.io/en/engineered-systems/health-diagnostics/trace-file-analyzer/tfaug/troubleshoot-tfa.html#GUID-169D2468-008B-4CE1-AB8E-1BA2A6233360

__________________
Troubleshooting TFA 
__________________

https://xy2401-local-doc-java.github.io/en/engineered-systems/health-diagnostics/trace-file-analyzer/tfaug/troubleshoot-tfa.html#GUID-AEEC5C9E-00F1-44B7-B39F-76E836AFC10F



##################################################################################
Oracle Autonomous Health Framework (Former TFA)
##################################################################################

Oracle Autonomous Health Framework is a collection of components that analyzes the diagnostic data collected, and proactively identifies issues before they affect the health of your clusters or your Oracle Real Application Clusters (Oracle RAC) databases. Oracle Autonomous Health Framework contains Oracle ORAchk, Oracle EXAchk, and Oracle Trace File Analyzer.

Install Oracle Autonomous Health Framework as root to obtain the fullest capabilities. Oracle Autonomous Health Framework has reduced capabilities when you install it as a non-root user.AFH can be run in two different modes either as daemon or Non-daemon mode. Both are doing same thing but daemon mode is more preferred.

To install Oracle AHF we run the ahf_setup installer with the -extract parameter. We also specify the -notfasetup parameter to avoid enabling the Oracle Trace File Analyser component of Oracle AHF.

For Rac  we need to install on both nodes . AHF will automatically synchronize between nodes.  IF show status  shows both nodes means   its synchronized no need to manually synchronize like tfa  

By default Oracle AHF will be installed to the /opt/oracle.ahf directory.

1) Download latest TFA software DocID 2550798.1
2) copy /u01/src/AHF-LINUX_v20.2.0.zip
3) unzip AHF-LINUX_v20.2.0.zip
4) Install  TFA  
as root 
 cd /u01/src/
 ./ahf_setup
[root@dbhost]# ./ahf_setup -extract -notfasetup 


 as  non root 
./ahf_setup -ahf_loc $ORACLE_HOME/ahf




__________________

upgrade the AHF 

__________________

1  mkdir /tmp/AHF

2  copy the latest AHF to /tmp/AHF and unzip 

4. uninstall existing  TFA : 

tfactl uninstall

3  ./ahf_setup -data_dir /u01/app/grid_base -tmp_loc /u01/app/grid_base/tmp/   ( You can find the data directory " ps -ef  | grep tfa | grep HeapDumpPath" ) 

4) Verify  TFA installed : 

/opt/oracle.ahf/tfa/bin/tfactl status

/opt/oracle.ahf/tfa/bin/tfactl toolstatus

5) syncnodes should be auto . below is just for reference 

tfactl syncnodes

Reference : 

Remove existing AHF and install latest AHF 21.4.1 as per MOS Doc ID 2832630.1

__________________
Controlling TFA  cpu  usage 
__________________

ahfctl setresourcelimit 

[-tool tool_name] 

[-resource resource_type] 

[-value value]

__________________
uninstall AHF 
__________________

[root@dbhost]# cd /opt/oracle.ahf/tfa/bin 
[root@dbhost]# ./tfactl uninstall 



__________________
Upload Files directly to Oracle Sr 
__________________

There are options to upload files  directly to  Oracle Sr 

$ curl -T [FILE_YOU_WANT_TO_SEND] -u [MOS_USER]

https://transport.oracle.com/upload/issue/[SR_NUMBER]/

$ tfactl upload -sr [SR_NUMBER] -user [MOS_USER] [FILE_YOU_WANT_TO_SEND]

__________________
AHF Reference 
__________________

https://docs.oracle.com/en/database/oracle/oracle-database/19/atnms/troubleshoot-tfa.html#GUID-11964D53-74C9-4754-9E80-9DB22557FF4E

https://docs.oracle.com/en/database/oracle/oracle-database/18/atnms/tfa-service.html#GUID-C470800D-B690-45F2-8C38-8EC60B6BB828

https://docs.oracle.com/en/engineered-systems/health-diagnostics/trace-file-analyzer/tfaug/performing-custom-collections.html#GUID-E4A2492E-A123-480A-B954-57898DBCE8BE