How to resize celldsik

 Leaving unallocated space on your celldisks instead of consuming 100% of it during initial griddisk creation is a standard Exadata best practice. It gives you an infrastructure safety net and operational flexibility.

​Here are the primary benefits of leaving that space behind and exactly how you can use it later:

​1. The Core Benefits of Leaving Headroom

​Emergency Overhead for Rebalancing: If a physical disk or an entire storage cell fails, ASM immediately shifts into survival mode. It copies the missing mirrored data blocks from the failed node onto the remaining healthy drives to restore full 3-way HIGH redundancy. Having empty space already sitting at the block layer gives the storage cluster immediate, unconstrained room to execute this emergency rebalance without running out of space.

​Avoiding "Disk Full" Lockups: If an ASM disk group fills up completely to 100%, the entire database freezes. Leaving unallocated space at the griddisk layer means you have an emergency reservoir of raw space hidden below the ASM layer that you can tap into within minutes to rescue a hung database.

​Workspace for Flash/Cell Diagnostics: The Exadata operating system uses unallocated areas for core dumps, cell internal tracing, and software upgrade image staging without encroaching on database data files.

​2. How That Space Can Be Used Later

​If your business needs change six months down the line, that unallocated space can be put to work in two ways:

​Scenario A: Dynamically Extending Your Existing Disk Groups

​If your +DATA01 disk group is running out of space, you don't need to purchase new hardware. You can create new, small "extension" griddisks out of that unallocated pool and add them straight to the existing group.

https://www.pythian.com/blog/how-to-resize-exadata-grid-disks-without-an-outage

as per this exmaple 

https://www.pythian.com/blog/how-to-resize-exadata-grid-disks-without-an-outage , as per this blog why only diskgrourp resize needed for DATA ? but not for RECO ?

As soon as that happens, Oracle ASM immediately detects that the underlying OS-level devices have shrunk. In modern Grid Infrastructure versions, ASM automatically adjusts its internal metadata for the RECO diskgroup to match the new, smaller size. If you look closely at the blog's output after the cellcli shrink but before any ASM command is run, v$asm_diskgroup already shows RECO dropped from 8,877 GB down to 2,100 GB.

Because ASM self-adjusts downwards when the storage layer shrinks, running an explicit ALTER DISKGROUP RECO RESIZE ALL is redundant.

When you expand DATA via cellcli, the underlying griddisks grow physically, creating new free sectors. However, ASM will never automatically consume newly available space on its own.

The Risk: If you ever try to shrink RECO at the cellcli layer when it doesn't have enough free space, or if the data blocks inside ASM are sitting at the end of the disks, cellcli will fail or you will risk corrupting the diskgroup because you are cutting off the disk where data blocks live.

If you were doing this in a tight production environment, the safest, most robust sequence is actually to do the reverse of the automatic behavior:

1. Shrink ASM First: Run ALTER DISKGROUP RECO RESIZE ALL SIZE 25G; to force ASM to safely rebalance and evacuate all data blocks out of the tail end of the disks.

2. Shrink Cell Second: Run cellcli to shrink the RECO griddisks.

3. Expand Cell Third: Run cellcli to expand the DATA griddisks.

4. Expand ASM Last: Run ALTER DISKGROUP DATA RESIZE ALL; to make ASM claim the new space.

5. 
   

Minimum DG size (MB) = ( USED_MB + GROWTH_MB ) * 1.15 

https://docs.oracle.com/en/engineered-systems/exadata-database-machine/sagug/resize-grid-disks-find-space.html

another exadata disk creation examle

 ok lets consider now I have 10 full capacitry storage server  , each provides 264 TB disk capacity 

how to create asm disk out of it , using dcli command ?

like I need to create celldsik then gridisk and then askdisk

I needs to create one +DATA01 and +RECo01 diskgorup with high reducnccy and needs to reserver some space as well 

show me the step by step and give me how does each stage looks after we running command like "cellcli -e CREATE CELLDISK ALL HARDDISK"

also explain as per industry standard like creating flashlog with specific size and then the flashlog 

as we have 27.2 TB Total Raw Smart Flash Cache / Log  per cell 

the 22 TB per drive (which gives you that $12 \times 22 \text{ TB} = 264 \text{ TB}$ raw total per cell) is not 100% useful for actual database storage due to binary formatting conversion, system OS partitions etc

Your raw 22 TB hard drive shows up to the Exadata operating system as roughly 20.01 TiB.

Stage                     | Math Formula / Process            | Total Cluster Space

--------------------------+-----------------------------------+--------------------

Raw Factory Space         | 10 cells x 264 TB                 | 2,640 TB

After Binary Conversion   | True OS recognized space (~91%)   | 2,402 TB

Minus Spare Cushion (10%) | Leaving space for cell rebuilds   | 2,162 TB

High Redundancy Mirroring | Divide by 3 (2,162 / 3)           | 720.6 TB

The Final Verdict

Out of your 2,640 TB of raw hardware space, your databases will see roughly 720 TB of total usable space shared between +DATA01 and +RECO01.

To configure 10 High-Capacity (HC) Exadata Storage Servers from scratch using dcli, you must follow a precise execution order. As an industry standard, you configure the flash performance layers first before touching the spinning disks (hard disks) to prevent space allocation conflicts.

Here is the step-by-step enterprise workflow to build your architecture, reserving space for high redundancy +DATA01 and +RECO01 disk groups.

The Strategic Sequence (Overview)

1. Flash Layer Configuration: Create Flash Log (512 MB per flash drive) $\rightarrow$ Allocate remaining flash to Flash Cache.

2. Physical to Logical Hard Disk Mapping: Create Celldisks from the 12 raw HDDs per cell.

3. Slicing Storage (Griddisks): Carve out precise per-disk sizes for DATA01 and RECO01 to leave a 10% unallocated cushion.

4. ASM Provisioning: Discover devices and build High Redundancy Disk Groups.

Step 1: Configure Flash Log & Flash Cache

Each cell has 4 physical Flash Cards. Industry standard dictates provisioning a small, highly effective 512 MB slice per flash disk for the log first, then letting the cache take the rest.

Bash

# 1. Creates a 512MB log slice per internal flash disk
dcli -g cell_group -l root "cellcli -e CREATE FLASHLOG ALL"

# 2. Allocates 100% of the remaining ~27.15 TB per cell to Flash Cache
dcli -g cell_group -l root "cellcli -e CREATE FLASHCACHE ALL"

How the stage looks:

• list flashlog shows 4 logs per cell, sized at 512M each, status normal.

• list flashcache shows a unified logical caching layer covering the remainder of the 27.2 TB, status normal.

Step 2: Create Celldisks from Hard Disks

Initialize the 12 physical spinning hard drives per cell.

Bash

dcli -g cell_group -l root "cellcli -e CREATE CELLDISK ALL HARDDISK"

How the stage looks:

Running cellcli -e list celldisk outputs the 12 mapped hard drives per cell:

Plaintext

CD_00_dm01cel01   normal
CD_01_dm01cel01   normal
...
CD_11_dm01cel01   normal

What it did: It cleared any old metadata and created an Exadata logical hardware abstraction over the raw blocks. True OS-recognized space per 22 TB drive drops to roughly 20.01 TiB ($\approx$ 22,000 GB) due to binary formatting math.

Step 3: Create Griddisks (Slicing per Hard Disk)

Because cellcli acts on individual drives, the sizing parameter must be specified per disk, not per cluster.

To achieve a 70% DATA / 20% RECO / 10% Free Cushion split based on the raw 22 TB nominal capacity:

• DATA01 Size per disk: $22 \text{ TB} \times 0.70 = \mathbf{15.4\text{T}}$

• RECO01 Size per disk: $22 \text{ TB} \times 0.20 = \mathbf{4.4\text{T}}$

• Remaining Cushion: Leaves exactly $2.2 \text{ TB}$ unallocated per disk ($\approx 10\%$).

Execute the slices sequentially. Creating DATA01 first guarantees it lands on the outer tracks of the platter for maximum IOPS:

Bash

# 1. Create DATA Griddisks on the high-performance outer tracks
dcli -g cell_group -l root "cellcli -e CREATE GRIDDISK ALL HARDDISK PREFIX=DATA01, size=15.4T"

# 2. Create RECO Griddisks right behind them
dcli -g cell_group -l root "cellcli -e CREATE GRIDDISK ALL HARDDISK PREFIX=RECO01, size=4.4T"

How the stage looks:

Running cellcli -e list griddisk will show 240 active griddisks across your 10 cells (24 per cell):

Plaintext

DATA01_CD_00_dm01cel01   active
RECO01_CD_00_dm01cel01   active
...

Step 4: Form the ASM Disk Groups (Run from DB Node)

Log into your primary Database Node as the grid user and access SQL*Plus (sqlplus / as sysasm).

ASM uses the Exadata network protocol (o/) to automatically aggregate the 120 DATA01 griddisks and 120 RECO01 griddisks into their respective disk groups.

SQL

-- 1. Create the +DATA01 Disk Group (High Redundancy)
CREATE DISKGROUP DATA01 HIGH REDUNDANCY 
  DISK 'o/*/DATA01_*' 
  ATTRIBUTE 
    'au_size'='4M',
    'compatible.asm'='19.0.0.0.0',
    'compatible.rdbms'='19.0.0.0.0',
    'cell.smart_scan_capable'='TRUE';

-- 2. Create the +RECO01 Disk Group (High Redundancy)
CREATE DISKGROUP RECO01 HIGH REDUNDANCY 
  DISK 'o/*/RECO01_*' 
  ATTRIBUTE 
    'au_size'='4M',
    'compatible.asm'='19.0.0.0.0',
    'compatible.rdbms'='19.0.0.0.0',
    'cell.smart_scan_capable'='FALSE';

Final Storage Cluster Profile

Because you specified High Redundancy (3-way mirroring) to protect your enterprise data, your usable database footprint calculates as follows:

Plaintext

Stage                     | Math Formula / Process            | Total Cluster Space
--------------------------+-----------------------------------+--------------------
Raw Factory Space         | 10 cells x 12 drives x 22 TB      | 2,640.0 TB
After Binary Conversion   | True OS recognized space (~91%)   | 2,402.4 TB
Targeted Sliced Space     | Sliced via 15.4T + 4.4T per disk  | 2,376.0 TB
Minus 10% Cushion Space   | Left unallocated for rebalancing  |   264.0 TB
High Redundancy Usable    | Divide Sliced Space by 3          |   792.0 TB

(Out of the 792 TB total usable cluster space, 616 TB belongs to +DATA01 and 176 TB belongs to +RECO01)

Explaining the Industry Standard Elements:

• o/*/DATA01_*: This is the Exadata internal discovery string syntax. The o/ tells ASM to use the Exadata specialized network protocol (o通用) to talk directly to cellsrv over InfiniBand/RoCE, bypassing standard Linux OS device files.

• HIGH REDUNDANCY: Ensures that every single block of your data is written simultaneously to 3 distinct storage cells. If any 2 storage servers completely lose power or catch fire at the same moment, your database stays online without data loss.

• cell.smart_scan_capable='TRUE': This flag tells the Exadata cells to offload predicate evaluation, column projection, and decryption directly to the cell CPUs for +DATA01, which is the cornerstone of Exadata's extreme query performance.

asm disk architecture

 ok lets consider now I have 3 full capacitry storage server  , each provides 264 TB disk capacity 

how to create asm disk out of it , using dcli command ?

like I need to create celldsik then gridisk and then askdisk

I needs to create one +DATA01 and +RECo01 diskgorup with high reducnccy and needs to reserver some space as well 

To provision your 3 High Capacity (HC) storage servers from scratch, you do not use dcli for the final ASM group creation—ASM disk groups are always created via standard SQL inside the database instance.

However, you use dcli to log into all 3 storage cells simultaneously to format the physical drives, build the celldisks, slice them into griddisks, and reserve your space.

Here is the exact step-by-step production workflow to build +DATA01 and +RECO01 using HIGH Redundancy (3-way mirroring) across your 3 cells.

The Infrastructure Plan (3 Cells)

• Physical Layout: 3 cells × 12 hard drives = 36 physical disks (Total: 792 TB Raw).

• Space Reservation Goal: We will allocate 70% of the space to DATA01, 20% to RECO01, and leave 10% unallocated as free headroom for safe cell rebalancing and diagnostic workspace.

Step 1: Create Celldisks on All 3 Cells

Log onto a database compute node as the celladmin or root user. Run this dcli command to format all 36 physical hard drives across your 3 cells (cell01, cell02, cell03) into usable celldisks:

Bash

dcli -g cell_group -l celladmin "cellcli -e CREATE CELLDISK ALL HARDDISK"

Verify it by listing them:

Bash

dcli -g cell_group -l celladmin "cellcli -e LIST CELLDISK"

Step 2: Create Griddisks with Space Reservations

Next, we slice those celldisks into logical griddisks. We will hard-code sizes to guarantee a clean split and leave the remaining trailing space completely unallocated.

Each physical drive is 22 TB (approximately 20.01 TiB / 20,490,000 MB of actual formatted OS space).

• DATA01 Split (70%): We assign 14,000,000 MB per disk.

• RECO01 Split (20%): We assign 4,000,000 MB per disk.

• Remaining Space (10%): Left unallocated.

Run the DATA01 Griddisk creation:

Bash

dcli -g cell_group -l celladmin "cellcli -e CREATE GRIDDISK ALL HARDDISK PREFIX=DATA01, size=14000G"

Run the RECO01 Griddisk creation:

Bash

dcli -g cell_group -l celladmin "cellcli -e CREATE GRIDDISK ALL HARDDISK PREFIX=RECO01, size=4000G"

Verify the layout on all cells:

Bash

dcli -g cell_group -l celladmin "cellcli -e LIST GRIDDISK WHERE availableSpace   != 0"

(You will see all 36 DATA01 and 36 RECO01 griddisks status as active).

Step 3: Create the ASM Disk Groups via SQL

Now that the griddisks exist at the storage layer, drop back to your database compute node. Log into your Grid Infrastructure environment as the grid user, access SQL*Plus as SYSASM, and run the SQL strings to build the disk groups.

Bash

. oraenv
# (Select +ASM1)
sqlplus / as sysasm

1. Create the +DATA01 Disk Group:

SQL

CREATE DISKGROUP DATA01 HIGH REDUNDANCY
  DISK 'o/*/DATA01_*'
  ATTRIBUTE 
    'au_size'='4M',
    'compatible.asm'='19.0.0.0.0',
    'compatible.rdbms'='19.0.0.0.0',
    'cell.smart_scan_capable'='TRUE';

2. Create the +RECO01 Disk Group:

SQL

CREATE DISKGROUP RECO01 HIGH REDUNDANCY
  DISK 'o/*/RECO01_*'
  ATTRIBUTE 
    'au_size'='4M',
    'compatible.asm'='19.0.0.0.0',
    'compatible.rdbms'='19.0.0.0.0',
    'cell.smart_scan_capable'='TRUE';

📊 Final Usable Space Metrics (3 Cells with HIGH Redundancy)

Because you are using HIGH Redundancy across exactly 3 cells, ASM places exactly 1 copy of every block on each of the 3 cells.

Disk Group	Total Raw Aggregated Size	Usable Database Space (Raw ÷ 3)
+DATA01	~492 TB Raw	~164 TB Usable Capacity
+RECO01	~140 TB Raw	~46.6 TB Usable Capacity

With this configuration, your data is fully protected. If any single storage cell goes down or suffers a total power loss, the remaining 2 cells continue running your databases without a single byte of data loss.

in eahc step pleae show me how it will show the output like list physicaldisk , list cell disk , initially list celldisk what it will show etc

Here is the complete visual execution walkthrough showing exactly how the outputs look inside the terminal at every stage of the provisioning process.

Step 1: Checking the Initial State of Raw Storage

Before any configuration is made, you use dcli to verify that the 3 storage cells see the factory-fresh physical hard drives. Initially, checking for celldisks or griddisks will return empty results.

Command: List Physical Disks

Bash

dcli -g cell_group -l celladmin "cellcli -e LIST PHYSICALDISK WHERE diskType=HardDisk"

Expected Output:

Plaintext

cell01: 20:0  STATUS=normal
cell01: 20:1  STATUS=normal
... [truncated lines for drives 2 through 10] ...
cell01: 20:11 STATUS=normal
cell02: 20:0  STATUS=normal
cell02: 20:1  STATUS=normal
...
cell03: 20:11 STATUS=normal

Command: List Celldisks (Initial State)

If you check for celldisks right now, the cells have no logical layout mapped.

Bash

dcli -g cell_group -l celladmin "cellcli -e LIST CELLDISK"

Expected Output:

Plaintext

cell01: 
cell02: 
cell03: 

(The output returns completely blank because no celldisks exist yet).

Step 2: Creating and Verifying Celldisks

This step initializes the physical disks into an Exadata-managed software layer.

Command: Create Celldisks

Bash

dcli -g cell_group -l celladmin "cellcli -e CREATE CELLDISK ALL HARDDISK"

Expected Output:

Plaintext

cell01: Celldisk CD_00_cell01 successfully created
cell01: Celldisk CD_01_cell01 successfully created
...
cell03: Celldisk CD_11_cell03 successfully created

Command: List Celldisks (After Creation)

Now when you query the cells, you will see all 36 initialized celldisks across the cluster (12 per cell).

Bash

dcli -g cell_group -l celladmin "cellcli -e LIST CELLDISK"

Expected Output:

Plaintext

cell01: CD_00_cell01  normal
cell01: CD_01_cell01  normal
...
cell02: CD_11_cell02  normal
...
cell03: CD_11_cell03  normal

Step 3: Creating and Verifying Griddisks (Slicing the Storage)

Now we carve up those 12 celldisks per cell into DATA01 and RECO01 fractions, keeping 10% unallocated.

Command: Create DATA01 Griddisks (70% Allocation)

Bash

dcli -g cell_group -l celladmin "cellcli -e CREATE GRIDDISK ALL HARDDISK PREFIX=DATA01, size=14000G"

Expected Output:

Plaintext

cell01: Griddisk DATA01_CD_00_cell01 successfully created
cell01: Griddisk DATA01_CD_01_cell01 successfully created
...
cell03: Griddisk DATA01_CD_11_cell03 successfully created

Command: Create RECO01 Griddisks (20% Allocation)

Bash

dcli -g cell_group -l celladmin "cellcli -e CREATE GRIDDISK ALL HARDDISK PREFIX=RECO01, size=4000G"

Expected Output:

Plaintext

cell01: Griddisk RECO01_CD_00_cell01 successfully created
cell01: Griddisk RECO01_CD_01_cell01 successfully created
...
cell03: Griddisk RECO01_CD_11_cell03 successfully created

Command: List Final Griddisk Layout

To confirm the sizing and see the remaining free space left on the drives, run a detailed attributes check:

Bash

dcli -g cell_group -l celladmin "cellcli -e LIST GRIDDISK attributes name, size, status"

Expected Output:

Plaintext

cell01: DATA01_CD_00_cell01   14000G   active
cell01: RECO01_CD_00_cell01   4000G    active
cell01: DATA01_CD_01_cell01   14000G   active
cell01: RECO01_CD_01_cell01   4000G    active
...
cell03: DATA01_CD_11_cell03   14000G   active
cell03: RECO01_CD_11_cell03   4000G    active

Step 4: Verification from the ASM DB Node

Finally, you move away from the storage cell CLI and log onto your database instance to confirm that Grid Infrastructure discovers your new disks as clean candidates.

Command: Query Candidate Disks inside SQL*Plus

SQL

SELECT path, header_status, os_mb FROM v$asm_disk WHERE header_status = 'CANDIDATE' ORDER BY path;

Expected Output:

Plaintext

PATH                     HEADER_STATUS      OS_MB
------------------------ -------------- ---------
o/192.168.10.11/DATA01_0 CANDIDATE       14336000
o/192.168.10.11/RECO01_0 CANDIDATE        4096000
o/192.168.10.12/DATA01_0 CANDIDATE       14336000
...
o/192.168.10.13/RECO01_11 CANDIDATE       4096000

72 rows selected.

(You will see exactly 72 paths: 36 for DATA01 and 36 for RECO01, all showing a status of CANDIDATE, meaning they are ready for your CREATE DISKGROUP commands).