OLTP on the NVM SDV: YMMV - H
Short Description
Download OLTP on the NVM SDV: YMMV - H...
Description
OLTP on NVM:
YMM
@andy_pavl o
The Last Six Months PDL Retreat October 2013
?
PDL Visit Day May 2014
GO OD
Prison Life
Washing Dishes Not Fighting
EVI L
Cafeteria Thievery Shankings
NVM OLTP
DRA M
Lightweight CC Logical
SSD/HD D Heavyweight CC ARIES
Overview
• Understand the performance characteristics of NVM to develop an optimal DBMS architecture for OLTP workloads. 9
OLTP Workloads
• Transactions have three defining characteristics: – Short-lived. – Small footprint. – Repetitive. 10
Intel NVM Emulator
• Instrumented motherboard that slows down access to the memory controller. • Two execution interfaces: – NUMA (NVM-only) – PMFS (DRAM+NVM) 11
NUMA Interface – NVM-Only
• Virtual CPU where all memory access uses the NVM portion of DRAM. • No change to application code. 12
PMFS Interface – DRAM+NVM
• Special filesystem designed for byteaddressable NVM. • Avoids overhead of traditional filesystems. 13
DBMS Architectures
• Disk-oriented. • Main memory-oriented.
14
Disk-oriented DBMS
• Pessimistic assumption that the data a txn needs is not in memory • Based on the design assumptions made in the 1970s. – Ingres (Berkeley)
15
Application
Buffer Pool
Primary Storage
ARIES LOG
Memory-oriented DBMS
• Assume that all data fits in memory. Avoid the overhead of concurrency control + recovery. – SmallBase (AT&T) – Hekaton (Microsoft) – H-Store/VoltDB (Me &
17
Application
Primary Storage CMD LOG
Snapshots
Experimental Evaluation
• Compare the DBMS architectures on the two NVM interfaces. • Yahoo! Cloud Serving Benchmark: – 10 million records (~10GB) – 8x database / memory
19
Evaluated Systems • NVM-Only
– H-Store (v2014) – MySQL (v5.5)
• NVM+DRAM – H-Store + Anti-Caching
(v2014) – MySQL (v5.5)
20
YCSB //
NUMA Interface (NVM-Only) Read-Only Workload 2x Latency Relative to DRAM
H-Store
MySQL
200,000
150,000 100,000 50,000
0 1 TXN/SEC
2 3 4 SKEW AMOUNT (HIGH→LOW)
5 21
YCSB //
PMFS Interface (NVM+DRAM) Read-Only Workload 2x Latency Relative to DRAM
Anti-Caching
MySQL
200,000
150,000 100,000 50,000
0 1 TXN/SEC
2 3 4 SKEW AMOUNT (HIGH→LOW)
5 22
YCSB //
NUMA Interface (NVM-Only) Write-Heavy Workload 2x Latency Relative to DRAM
H-Store
MySQL
50,000 40,000 30,000
20,000 10,000
0 1 TXN/SEC
2 3 4 SKEW AMOUNT (HIGH→LOW)
5 23
YCSB //
PMFS Interface (NVM+DRAM) Write-Heavy Workload 2x Latency Relative to DRAM
Anti-Caching
MySQL
50,000 40,000 30,000
20,000 10,000
0 1 TXN/SEC
2 3 4 SKEW AMOUNT (HIGH→LOW)
5 24
Discussion
• NVM latency did not make a big difference in performance. • Logging is major bottleneck in DBMS performance on NVM. • MySQL wastes NVM space.
25
Database Research @
CMU
Database Research @ CMU • • • •
N-Store H-Store + Anti-Caching ThomasDB MongoDB 27
N-STORE
N-Store
• NVM-only Architecture. • Hybrid OLTP/OLAP DBMS: – High-performance txn
processing. – Low-latency analytical operations. – Instant recovery.
29
N-Store – Shadow Paging 1 2 3 4
X
Master Page Table
X X X
DB Root 1 2 3 4
Shadow Page Table 30
H-STORE
H-Store + Anti-Caching
• Allows DBMS to mange DBs that are larger than amount of DRAM. • Reducing memory overhead of evicted data. • Using ML to reduce disk I/O.
32
MAIN TUPLE TABLE
{
Ø
EVICTED TUPLE TABLE
HEADER
LRU INDEXES
ANTI-CACHE
CHAIN
ANTI-CACHE BLOCKS 33
H-Store + Anti-Caching
• Bloom filter usage tracking. • Multi-faceted indexes. • Cold storage approximations. • Semantic block clustering.
34
ThomasDB
ThomasDB
• High-performance, lowoverhead incremental computation platform. • Maintains mapping between function invocations and data. 36
ThomasDB – Data Marts PUBLIC DATABASE
PRIVATE DATABASE
Update!
UDF
UDF
⨝
Materialized View
ANALYTICS PROGRAM
37
ThomasDB – Preliminary Results MAP FUNCTION OVER WIKIPEDIA CORPUS
38
MongoDB
Distributed Document Database Design • Automatic tool selects the near-optimal physical design: – Sharding Keys – Denormalization – Indexes 40
Vigilante DBA
• Automated framework for finding DB applications and fixing them. • Build a large catalog of database applications from public sources. 41
Summary
• Lots of database stuff in the works. • Always looking for industry collaborators.
42
END
@ANDY_PAVLO db.cs.cmu.edu
This talk is in compliance with the Federal Bureau of Prisons (FBP) Act (1930) Pub. L. No. 71-218, 46 Stat. 325. All reference herein to any specific commercial products, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the FBP. The views and opinions of authors expressed herein do not necessarily state or reflect those of the FBP, and shall not be used for evaluation by including, but not limited, to FBP parole board, its directors, and other corrections officers. Regulations are contained in Title 28, Chapter V of the Code of Federal Regulations (CFR). Contact with parolees for research questions is regulated under 28 C.F.R. 540. All complaints about the
View more...
Comments