From Oracle 12.1.0.2 Start , have access to asmfd To replace udev Under the rules asm Disk device binding , At the same time, he also has the ability to filter illegal IO Characteristics of operation .
Check that the operating system version supports ASMFD, You can use the following code ：
acfsdriverstate -orahome $ORACLE_HOME supported
In redhat or centos 7.4 or above, you need to upgrade kmod to enable AFD
If we are planning to Upgrade from Asmlib to Afd we can refer (Doc ID 2172754.1)
Installation Steps :
Step 1> Stop Crs
Step 2> Configure AFD (ASM Filter Driver)
To check whether the operating system version supports ASMFD, you can use the following code:
acfsdriverstate -orahome $ORACLE_HOME supported
asmcmd afd_configure
Step 3> Configure Disk Discovery for AFD
Modify the below mentioned files as shown
cat /etc/afd.conf
afd_diskstring='/dev/xvd*'
afd_filtering=enable
cat /etc/oracleafd.conf
afd_diskstring='/dev/xvd*'
afd_filtering=enable
At this point AFD is configured you can verify the status as below
asmcmd afd_state
ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host 'wdtest05'
I would recommend to restart acfs before you proceed
acfsload stop
acfsload start
lsmod | grep acfs
Or by using dsset
[root@rac1 ~]# asmcmd dsset '/dev/asm*','AFD:*'
[root@rac1 ~]# $ORACLE_HOME/bin/asmcmd dsget
parameter:/dev/asm*, AFD:*
profile:/dev/asm*,AFD:*
Step5> Label the Disks using AFD to be used by ASM
#asmcmd afd_label DATA01 /dev/xvhg
#asmcmd afd_label FRA01 /dev/xvhh
#asmcmd afd_label REDO01 /dev/xvhi
#asmcmd afd_lsdsk
asmcmd afd_dsget
Checking Storage Multipath information :
[root@rac1 yum.repos.d]# ll /dev/mapper/mpath*
lrwxrwxrwx 1 root root 7 Feb 15 17:18 /dev/mapper/mpathc -> ../dm-1
lrwxrwxrwx 1 root root 7 Feb 15 17:18 /dev/mapper/mpathd -> ../dm-0
The multipath devices mpathc and mpathd are used here
[root@rac2 ~]# multipath -ll
mpathd (14f504e46494c45526147693538302d577037452d39596459) dm-1 OPNFILER,VIRTUAL-DISK
size=30G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 34:0:0:1 sdc 8:32 active ready running
|-+- policy='service-time 0' prio=1 status=enabled
| `- 35:0:0:1 sde 8:64 active ready running
|-+- policy='service-time 0' prio=1 status=enabled
| `- 36:0:0:1 sdg 8:96 active ready running
`-+- policy='service-time 0' prio=1 status=enabled
`- 37:0:0:1 sdi 8:128 active ready running
mpathc (14f504e46494c45524f444c7844412d717a557a2d6b7a6752) dm-0 OPNFILER,VIRTUAL-DISK
size=40G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 34:0:0:0 sdb 8:16 active ready running
|-+- policy='service-time 0' prio=1 status=enabled
| `- 35:0:0:0 sdd 8:48 active ready running
|-+- policy='service-time 0' prio=1 status=enabled
| `- 36:0:0:0 sdf 8:80 active ready running
`-+- policy='service-time 0' prio=1 status=enabled1
`- 37:0:0:0 sdh 8:112 active ready running
Moving disk from one diskgroup to another when using asm filter and multipath :
1) Drop disk from asm ( example ASM_DISKGROUP1_11 )
2) afdtool -delete ASM_DISKGROUP1_11 ( on one node by Unix team using root )
3) afdboot -scandisk / -rescan ( on all nodes by Unix team using root )
4) remap ASM_DISKGROUP1_11 to new disk name in /etc/multipath.conf , relabel disk and reload multipath ( on all nodes by Unix team using root )
To reload Mulipath service : service multipathd restart
5) Wait for disk to be visible in /dev/mapper on all nodes
afdtool -add /dev/mapper/ASM_DISKGROUP1_12 ( on one node by Unix team using root )
6) afdboot -scandisk / -rescan ( on all nodes by Unix team using root )
afdtool -getdevlist
7) asmcmd afd_lsdsk
Known Issue 1 )
AFD: AFD Is Not Loaded Automatically After Node Reboot Due To Incorrect Dependencies (Doc ID 2724726.1)
Bug 31771370 - INCORRECT OHASD/AFD SERVICE DEPENDENCIES AT OS LEVEL AFTER OS UPDATE TO LINUX 7.8 AFFECTING AFD AND/OR CLUSTERWARE STARTUP
Solution we did was to rescan afd_scan
. oraenv
+ASM4
crsctl stop crs -f
asmcmd afd_state
acfsload stop # stop acfs driver stack
afdload stop # stop acfsd driver
asmcmd afd_scan # scan the devices
acfsload start # start acfs driver stack
asmcmd afd_lsdsk # list asm disks
asmcmd afd_filter -e # enable ASM filter
asmcmd afd_state
crsctl start crs -wait # start crs
$GRID_HOME/bin/afdload start # start acfsd driver
/usr/bin/afdboot -scandisk
$GRID_HOME/bin/crsctl stop crs -f
$GRID_HOME/bin/crsctl start crs
We same issue was faced after server migration we also executed below before enable ASM filter
asmcmd afd_configure
References :
https://docs.oracle.com/database/121/OSTMG/GUID-302F7233-B905-4D1E-A1AD-9A00EDC6A6F3.htm#OSTMG95732
ASMFD : How to Migrate ASM Diskgroups from ASMLIB to ASMFD (ASM Filter Driver) on Oracle Grid Infrastructure (RAC) (Doc ID 2172754.1)
No comments:
Post a Comment