January 6, 2018 | Author: Anonymous | Category: Engineering & Technology, Computer Science, Data Management
Share Embed Donate

Short Description

Download Data_Warehoue_Overview_20141010...


3SAQS Data Warehouse Update October 10, 2014 CIRA - Fort Collins, CO

Outline • • • • • •

Data Warehouse Team System Overview Phase 1 Operational State Website Overview Project & Request Workflow Future Work

3SAQS Steering Committee October 10, 2014


Data Warehouse Team • • • • •

Shawn McClure Tom Moore Dustin Schmidt Rodger Ames Cassie Archuleta (ARS)

3SAQS Steering Committee October 10, 2014


System Overview

Monitoring Data

Modeling Platforms

IMPROVE, STN, Met, etc.


External modeling activities

PGM and AQ impacts assessment by consultant


ROD/EIS published by Agency Sponsor

Modeling platforms, monitoring data, completed modeling studies are uploaded along with related documentation.

Project Management

Delivery Modeling files for a specific request are obtained by the user. Each data request is accompanied by a manifest. After receipt, user signs off on request.

Project Manager/Data User initiates a project on the DW. Can assigns collaborators who can submit a data request.

Data Request

3SAQS Steering Committee October 10, 2014

User submits request within an active project. Request may be for pre-defined data package (current), or a la carte selection (future).


Data Warehouse – Phase 1 • Project Management System – Project/inventory tracking, modeling platform request form (Phase 1=NEPA), data manifest

• Hardware Upgrades – New 55tb RAID – USB3 transfer to external disks

3SAQS Steering Committee October 10, 2014


Data Warehouse – Phase 1 (cont.) • Modeling Platforms – Base 2008b, Base 2011a CAMx (release pending approval)

• Data deliveries – Hiawatha (ENVIRON) – Converse County (AECOM)

• Website – Pages for studies, projects, documents, procedures and results, meetings & events, literature, glossary, about (updates ongoing) 3SAQS Steering Committee October 10, 2014


Data Warehouse – Phase 1 (cont.) • Inventory Management – Data is placed within the warehouse – Warehouse file system is represented in database • Facilitates change control

– File system items are data • Files • Directories

– Categories are groups of File System Items • • • •

Files and Directories are grouped into Categories Meta data is attached to Category Categories may be hierarchically related Data may belong in multiple categories

– Data Packages are groups of Categories • Collection of categories for intended use (NEPA)

Website overview • Studies – Modeling platforms (completed and forward looking) – Monitoring data overview (pending)

• Documents – Modeling platform protocols, MPEs, etc.

• Procedures and Results – Specific how-to pages for DW web based tools (pending)

• About – General information about 3SAQS and the DW (pending)

• Projects – User initiated AQ modeling using 3SAQS platforms 3SAQS Steering Committee October 10, 2014


Demo Website

3SAQS Steering Committee October 10, 2014


Data Warehouse Website – Phase 1



3SAQS Steering Committee October 10, 2014

Project Management



3SAQS Steering Committee October 10, 2014


Studies 3SAQS foundational modeling platforms • Phase 1 (done) – 2008b available – 2011a (release pending)

• Phase 2 (forward looking) – 2008_2020a – 2011_2020a (CAMx/CMAQ) – Preliminary 2014 (CAMX/CMAQ) – 2014_FutureCase (CAMX/CMAQ) 3SAQS Steering Committee October 10, 2014


3SAQS Steering Committee October 10, 2014


Documents •

Memorialize work that is already done –

Reference basis for modeling platforms that the DW is delivering – – –

WestJump to 3SAQS 3SAQS and 2011 NEI O&G Emission Inventories reconciliation 3SAQS Methane inventory & response to comments

Modeling Guidance – –

• •

Base 2008b MPE 2011 WRF MPE

Technical memos – – –

2008, 2011 Modeling Protocols & response to comments

Calls for comments on draft documents – –

Modeling protocols MPE reports Emissions reconciliations

Reports and comments –

Network Assessment

NEPA/non-NEPA guidance SMOKE

Work plans Annual & Quarterly Reports 3SAQS Steering Committee October 10, 2014


3SAQS Steering Committee October 10, 2014


Procedures and Results • “How to” pages for: – Data download and upload procedures – Application of visualization tools – Understanding modeling platforms – Interpreting emissions inventories – Accomplishments and future work of the 3SAQS and DW

3SAQS Steering Committee October 10, 2014


About • Informational Flyer – Goals of 3SAQS and role of DW – Studies and project management overview – Modeling toolbox highlights • Geospatial data selection

• 3SAQS and cooperators – – – –

BLM NPS Forest Service States of CO, WY, UT 3SAQS Steering Committee October 10, 2014


Projects User initiated AQ modeling projects • NEPA Projects – Hiawatha (ENVIRON) – Converse County (AECOM) – Others pending

• Non-NEPA (future work) – – – – – –

NPS N deposition NPS Vertical Profiles of simulated ammonia EPA QA 2011 Base A State of CO RAQC O3 SIP State on NM O3 nonattainment Others… 3SAQS Steering Committee October 10, 2014


Project Workflow - High Level Goals • Manage Data Access • Provide Project Managers an overview of data delivery progress • Provide Users a concise view of available data • Provide Data Requestors with a detailed manifest (receipt) • Detailed data accounting (Inventory Management) – Meta data – Who got what and when – Change control

• Track usage of supplied data • Capture communication threads (Future work)

Project Workflow - Overview • 3SDW Administrators designate registered users as “Project Managers” – Cooperating Agency point of contact

• Project Manager creates new analysis project – Provides high level meta data – Designates project team members • Contractors, Subordinates, Etc

• Project team member initiates a Data Request – Data Selection – Provide detailed meta-data

• 3SDW Team fulfills data request • Project team member signs-off request as completed 3SAQS Steering Committee October 10, 2014


Project Workflow – User Roles

Project Workflow - User Roles • Registered User – View projects (Read-only) – View Data Requests (Read-only)

Project Workflow - User Roles • Project Manager – A Registered user who is also a member of a 3SDW cooperating agency (BLM, EPA, NPS, etc). – Create new analysis projects – Assign project team members – Monitor data transfer progress – Issue data requests

Project Workflow - User Roles • Project Team Member – Assigned to team by Project Manager – Issue data requests for assigned projects

Project Workflow

Projects Overview • Purpose – Present an overview of Data Request status – Entry point for Project process – Create a new project

Projects Overview

Projects Overview

Project Configuration • Purpose – Capture high level information about the project • High level description (Details provided by data requestors) • Location • Timeframe

– Managing Data Access • Selected pool of Project Managers (3SDW Cooperators) • Project Managers assign 3SDW registered users to project

Project Configuration

Project Configuration

Data Request • Allow users to browse available data • Capture optional information about – Intended use of data – Results/Conclusions from data usage

• Supply detailed manifest of requested data

Data Request

Data Request

Data Request

Data Request

Future Work • • • • •

Provide finer granularity in data selection Tune process for supplying supplemental data Web interface for Data Cataloging & QA Enhance performance of online data transfer Capture e-mail communication traffic for Project / Data Request • Data Submission Process (Return model results to warehouse)

Data Warehouse Website – Phase 2 Documents Sign in/ Register/user’s account information

> Software benchmarking > Modeling platforms > Modeling Protocol > MPE reports > Glossary > Modeling and website guidance

Contact us

Site admin


MPE plot browser

News and Events Documents

> Development milestones > Links

Project Management

Tools Emissions review tool

Literature O&G construction and production emissions Excel work books

> Articles on air quality modeling related to O&G developments in the western region Studies > High level view of foundational modeling platforms either available or pending in the DW

Projects Overview > status of projects > project access can be restricted to specific users/group

Project Configuration > create new projects > edit existing projects > view data request history

NEPA Data Request > Request form based on NEPA test case template > tabular or tree view file view with file categories and metadata

Manifest > data request manifest at directory and file level

Legend Stage 1: September 2014 deployment

3SAQS Steering Committee October 10, 2014


View more...


Copyright � 2017 NANOPDF Inc.