..: Create ASM DISKGROUP with EXADATA FLASHDISK

X7-2 half rack Exadata machine having 4 storage nodes and 24TB allocated to flashcache on each cell

We are going allocate 12TB flashcache and 12TB to flashdisk

Check flashcache details:

CellCLI> list flashcache detail

cellDisk: FD_00_dbm0celadm01,FD_02_dbm0celadm01,FD_01_dbm0celadm01,FD_03_dbm0celadm01

creationTime: 2019-01-19T19:57:14-08:00

degradedCelldisks:

effectiveCacheSize: 23.28692626953125T

id: 1241246b-af5c-4636-b4a1-d23f21334c66

size: 23.28692626953125T

status: normal

Will drop and recreate flashcache with 12TB:

CellCLI> drop flashcache

CELL-02769: Cannot complete flash cache drop on: FD_00_dbm0celadm01. Received error: CELL-02732: There may be data not synchronized to the grid disks (dirty data) in flash cache. Use ALTER FLASHCACHE to flush the dirty data and to stop caching, and then retry the command. Flash cache was not dropped on these cell disks: FD_00_dbm0celadm01 FD_02_dbm0celadm01 FD_01_dbm0celadm01 FD_03_dbm0celadm01 .

CellCLI> alter flashcache all flush

Monitor flush status from another session:

CellCLI> list flashcache detail

cellDisk: FD_00_dbm0celadm01,FD_02_dbm0celadm01,FD_01_dbm0celadm01,FD_03_dbm0celadm01

creationTime: 2019-01-19T19:57:14-08:00

degradedCelldisks:

effectiveCacheSize: 23.28692626953125T

id: 1241246b-af5c-4636-b4a1-d23f21334c66

size: 23.28692626953125T

status: normal - flushing

CellCLI> list flashcache detail

cellDisk: FD_00_dbm0celadm01,FD_02_dbm0celadm01,FD_01_dbm0celadm01,FD_03_dbm0celadm01

creationTime: 2019-01-19T19:57:14-08:00

degradedCelldisks:

effectiveCacheSize: 23.28692626953125T

id: 1241246b-af5c-4636-b4a1-d23f21334c66

size: 23.28692626953125T

status: normal - FD_01_dbm0celadm01 (flushing), FD_03_dbm0celadm01 (flushing), FD_00_dbm0celadm01 (flushed), FD_02_dbm0celadm01 (flushed)

CellCLI> list flashcache detail

cellDisk: FD_00_dbm0celadm01,FD_02_dbm0celadm01,FD_01_dbm0celadm01,FD_03_dbm0celadm01

creationTime: 2019-01-19T19:57:14-08:00

degradedCelldisks:

effectiveCacheSize: 23.28692626953125T

id: 1241246b-af5c-4636-b4a1-d23f21334c66

size: 23.28692626953125T

status: normal - flushed

Drop Existing flashcache:

CellCLI> drop flashcache

Flash cache dbm0celadm01_FLASHCACHE successfully dropped

CellCLI> list flashcache detail

Create flashcache with 12TB:

CellCLI> create flashcache all size=12288G

Flash cache dbm0celadm03_FLASHCACHE successfully created

CellCLI> list flashcache detail

cellDisk: FD_02_dbm0celadm03,FD_03_dbm0celadm03,FD_01_dbm0celadm03,FD_00_dbm0celadm03

creationTime: 2019-03-21T22:56:17-07:00

degradedCelldisks:

effectiveCacheSize: 12T

id: b14774c0-8d09-4cf3-ac15-51d88968e9cf

size: 12T

status: normal

Create flashdisk:

CellCLI> CREATE GRIDDISK ALL FLASHDISK PREFIX='FDOM'

GridDisk FDOM_FD_00_dbm0celadm03 successfully created

GridDisk FDOM_FD_01_dbm0celadm03 successfully created

GridDisk FDOM_FD_02_dbm0celadm03 successfully created

GridDisk FDOM_FD_03_dbm0celadm03 successfully created

CellCLI> list griddisk attributes diskType, name where diskType=FLASHDISK

FlashDisk FDOM_FD_00_dbm0celadm03

FlashDisk FDOM_FD_01_dbm0celadm03

FlashDisk FDOM_FD_02_dbm0celadm03

FlashDisk FDOM_FD_03_dbm0celadm03

CellCLI> list griddisk FDOM_FD_00_dbm0celadm01 detail

asmDiskGroupName:

asmDiskName:

asmFailGroupName:

availableTo:

cachedBy:

cachingPolicy: default

cellDisk: FD_00_dbm0celadm01

comment:

creationTime: 2019-03-21T23:02:52-07:00

diskType: FlashDisk

errorCount: 0

id: a649aff8-20ec-4622-9315-8d877e6705ab

size: 2.8217315673828125T

status: active

We need at least one failover group to create asm diskgroup in x7-2

Repeat same step on storage node 2 and create flashdisk

Set asm disk string parameter:

alter system set asm_diskstring='o/*/DATAC1_*','o/*/RECOC1_*','o/*/FDOM_*','/dev/exadata_quorum/*' scope=both sid='*';

Verify newly created flashdisk from cell nodes visible on asm or not:

SQL> select name, total_mb/1024, free_mb/1024, path, header_status from v$asm_disk where header_status='CANDIDATE';

NAME TOTAL_MB/1024 FREE_MB/1024 PATH HEADER_STATU

-------------- ------------- ------------ ----------------------------

0 0 o/000.00.00.15;000.00.00.16/FDOM_FD_00_dbm0celadm04 CANDIDATE

0 0 o/000.00.00.9;000.00.00.10/FDOM_FD_03_dbm0celadm01 CANDIDATE

0 0 o/000.00.00.15;000.00.00.16/FDOM_FD_01_dbm0celadm04 CANDIDATE

0 0 o/000.00.00.9;000.00.00.10/FDOM_FD_00_dbm0celadm01 CANDIDATE

0 0 o/000.00.00.13;000.00.00.14/FDOM_FD_03_dbm0celadm03 CANDIDATE

0 0 o/000.00.00.15;000.00.00.16/FDOM_FD_02_dbm0celadm04 CANDIDATE

0 0 o/000.00.00.9;000.00.00.10/FDOM_FD_02_dbm0celadm01 CANDIDATE

0 0 o/000.00.00.15;000.00.00.16/FDOM_FD_03_dbm0celadm04 CANDIDATE

0 0 o/000.00.00.11;000.00.00.12/FDOM_FD_01_dbm0celadm02 CANDIDATE

0 0 o/000.00.00.11;000.00.00.12/FDOM_FD_03_dbm0celadm02 CANDIDATE

0 0 o/000.00.00.13;000.00.00.14/FDOM_FD_01_dbm0celadm03 CANDIDATE

0 0 o/000.00.00.9;000.00.00.10/FDOM_FD_01_dbm0celadm01 CANDIDATE

0 0 o/000.00.00.13;000.00.00.14/FDOM_FD_02_dbm0celadm03 CANDIDATE

0 0 o/000.00.00.11;000.00.00.12/FDOM_FD_02_dbm0celadm02 CANDIDATE

0 0 o/000.00.00.13;000.00.00.14/FDOM_FD_00_dbm0celadm03 CANDIDATE

0 0 o/000.00.00.11;000.00.00.12/FDOM_FD_00_dbm0celadm02 CANDIDATE

16 rows selected.

Create asm diskgroup:

SQL> CREATE diskgroup FLASHDISK normal redundancy disk 'o/*/FDOM*'

attribute 'compatible.rdbms'='11.2.0.4.0',

'content.type' = 'data',

'compatible.asm'='18.0.0.0.0',

'cell.smart_scan_capable'='TRUE',

'au_size'='4M'; 2 3 4 5 6

Diskgroup created.

SQL> select name,state,total_mb/1024, free_mb/1024 from v$asm_diskgroup;

NAME STATE TOTAL_MB/1024 FREE_MB/1024

-------------- ----------- ------------- ------------

DATAC1 MOUNTED 61440.25 3746.0625

RECOC1 MOUNTED 16368 12160.2344

BACKUP MOUNTED 30720 2719.21875

FLASHDISK MOUNTED 46231.25 46230.3164

..

labels

Tuesday, 26 March 2019

Create ASM DISKGROUP with EXADATA FLASHDISK

No comments:

Post a Comment