ASM vs filesystem performance

Filesystem performance vs. ASM
ASM Performance vs.
Filesystems
Large amounts of available memory on the server
will always benefit filesystems, since the
filesystem driver caches recently used blocks in
virtual memory. Performance is directly affected,
since database physical I/Os can be serviced from
the filesystem buffer cache, thus providing a
faster I/O response time.
Problem:
Leveling the playing field:
•
•
•
Incorrect conclusions regarding ASM
performance drawn from invalid
benchmark comparisons.
Incorrect assumptions regarding
filesystem features lost when migrating to
ASM.
Incorrect assumption that Database I/O
performance is bounded by ASM.
•
•
Background:
•
•
•
•
•
ASM uses character or “unbuffered” I/O
interface (raw). Thus no I/O caching
occurs at the OS level.
Filesystems use a block or buffered I/O
interface (cooked). OS Virtual memory is
used extensively for I/O caching.
As a result of the different I/O code paths,
and the use of additional memory, a direct
comparison is invalid.
Filesystems employ performance features
that are not necessary for databases.
ASM is not in the database I/O code path.
When the database instance starts, ASM
sends the extent map to the database
instance, which is stored in the database
instance SGA. Hereafter all database I/O
is performed directly by the database
instance and does not go through ASM.
Thus database I/O is not bounded by
ASM.
Filesystems generally benefit from large OS
caches. When ASM is used, the benefits of
the extra layer of cache is lost. Thus
determine the amount of memory used by the
filesystem for the application being
benchmarked (working set size), and add that
to the database buffer cache or the
SGA_TARGET of the database.
Alternatively, if the filesystem has the
capability of supporting o_direct calls or
DIRECTIO setting, than that option should be
enabled when benchmarking against ASMbased databases. This technique is used to
eliminate the cache from the filesystem I/O
path.
This is the same technique that has been applied
to cooked vs. raw performance device
comparisons for many years. It is highly
recommended to use Oracle’s ORION utility to
determine I/O throughput of the underlying
ASM/filesystem LUNs.
Filesystem features and ASM
There are several strong virtues of filesystem, but
these benefits can also impact database
performance and create unnecessary management
overhead:
o Read ahead pre-fetch – This feature performs
anticipatory pre-fetches based on the
sequential access pattern of the session query.
Though generally this is a useful feature, it
does have CPU overhead to evaluate the “next
I/O” pattern, and sometimes will pre-fetch too
much, causing unnecessary congestion on the
I/O subsystem. This feature will also have
limited benefits if the filesystem is
fragmented or can become too aggressive
with small sequential accesses.
Filesystem pre-fetch is generally not needed,
since the Oracle Database has built-in prefetch capability. Additionally, large block
sequential I/Os are generally performed using
database Parallel Query (PQ) processing,
which is much more efficient than serialized
sequential scans.
PQ processing on a
filesystem is very inefficient since the I/O is
bounded by the throughput of the single OS
device; whereas PQ in an ASM configuration
will fan-out and leverage all the disks in the
diskgroup. With Exadata, the performance is
even further improved by minimizing the data
sent through I/O channels.
o Filesystem Write I/O coalescing – The
filesystem provides the capability to coalesce
several write requests to the same file into a
single write. This is a nice feature, but it has
a tendency to cause I/O write stalls.
This feature is also generally not needed since
most database writes are coalesced by DBWR
background process. DBWR has built-in
capability to coalesce dirty buffers, place
them in a queue and write them out as a large
set. This is the same for LGWR process,
except LGWR writes memory contiguous log
buffers. User process data loads are primarily
done via PQ DML slaves, thus write I/O
coalescing is inherently performed.
o Filesystem DirectIO – Allows un-buffered I/O
to filesystem files, mimicking raw device like
performance. However, all directIO must be
I/O aligned or they will fallback to standard
I/O. Additionally, directIO does not perform
well for every workload; e.g., small block I/O
applications may incur more overhead.
DirectIO is not the default for most
filesystems, thus this each application must be
monitored and modeled before directIO is
enabled. Furthermore, this is a manual change
to a filesystem. In some cases, enabling
directIO also implicitly disables the certain
other filesystem features, such as read prefetch.
With ASM, much of the filesystem
management, monitoring and tuning is
mitigated. Since ASM presents raw storage
to the database and the database in turn does
raw I/O directly to the device, it reaps all the
benefits of un-buffered I/O without having to
set any parameters or modify any mount
options. DirectIO is inherently provided by
ASM.
Conclusion:
The comparison of cooked vs. raw has been going
on for over a dozen years. The conclusion has
always been that raw devices will generally
provide better performance than filesystems;
however, manageability of raw makes it a
cumbersome choice. ASM provides best of both
worlds; i.e., performance of raw with
manageability of filesystems, with the additional
benefit of being optimized specifically for
database I/O patterns. Given equivalent operating
conditions, ASM will always perform at least as
well, if not better, than filesystems.

Download Report