Changes

Jump to: navigation, search

OpenZFS Developer Summit

15,520 bytes removed, 17:18, 23 April 2019
no edit summary
The ''OpenZFS Developer Summit'' was held November 18-19, 2013. The goals of the event were:* to foster cross-platform community discussions of OpenZFS work* to make progress on some of the projects proposed for this community. == Event Report == [https://www.youtube.com/playlist?list=PLaUVvul17xSdWMBt5tAC8Hu7bbeWskD_q Video recordings of the presentations] are in the OpenZFS channel on YouTube. === Topics that were discussed ===* Platform Panel (w/ representatives from illumos, FreeBSD, Linux, and OSX)* Platform-independent code repository (Matt Ahrens)* Storage tiering (Boris Protopopov)* Vendor Panel (all represented companies sharing their work)* Community planning (Karyn Ritter)* Channel programs (Chris Siden & Max Grossman)* Test coverage (John Kennedy)* Performance investigations (Adam Leventhal)* Performance on full/fragmented pools (George Wilson)* Scalability (Kirill Davydychev)* Virtual memory interactions (Brian Behlendorf)* Multi tenancy (Rob Mustacchi)* Examining on-disk format (Max Bruning) === Hackathon projects ===These projects were worked on at the hackathon: * Testrunner suite running on other platforms: linux, freebsd (+ illumos libzpool)* channel programs: pass instruction stream into kernel* ZFS send progress reporting with "set proc title"* dedicated kmem_cache for dedup table entries* larger (1MB) blocksize * compressed ARC* upstreaming dbuf_user & dbuf state machine (spectra DMU changes)* limit DDT size* onexit for osx   == A huge THANK YOU to our generous sponsors =={|cellpadding="10"|- |[[File: Delphix logo.png|200x200px|link=http://delphix.com]]|[[File: Ddrdrive logo dev 2013.png|200x200px|link=http://www.ddrdrive.com]] |[[File: Logo freebsdfoundation.png|200x200px|link=http://www.freebsd.org]] |-|[[FIle: SageCloud-logo-RGB-large-optimized.jpg|200x200px|link=http://sagecloud.com/]] |[[File: Ixsystems.jpg|200x200px|link=http://www.ixsystems.com/]] |[[File: CS logo withTagline 150px.png|200x200px|link=http://cloudscaling.com/]] |-|[[File: SpectraLogicLogo.jpg|200x200px|link=http://www.spectralogic.com/]] |[[File: Joyent-logo.png|200x200px|link=http://joyent.com]]|[[File: New_hac_logo.jpg|200x200px|link=http://www.high-availability.com/]]|-|[[File: Nexenta logo.jpg|200x200px|link=http://www.nexenta.com/]]|} = Event Planning Info = == Schedule ===== Sunday, November 17th ==={|border="1" cellpadding="10"|-|6:30pm| Optional dinner at [http://goo.gl/943i5i Rosamunde Sausage Grill] on Mission St.Location is a 3 minute walk from the 24th Street BART station.|} === Monday, November 18th ==={|border="1" cellpadding="10"|-|valign="top"|8:30am - 9:00am| Registration and breakfast|-|valign="top"|9:00am - 9:15am| Introductions|-|valign="top"|9:15am - 9:45am| State of OpenZFS & Event overview (Matt Ahrens)|-|valign="top"|9:45am - 10:00am| List and prioritize discussion topics (lead by Matt Ahrens & Karyn Ritter)|-|valign="top"|10:00am - 11:00am| Discussion sessions|-|valign="top"|11:00am - 11:15am| Coffee & snack break|-|valign="top"|11:15am - 12:15pm| Discussion sessions|-|valign="top"|12:15am - 1:00pm| Lunch|-|valign="top"|1:00pm - 1:30pm| Discussion session: Community Management (lead by Karyn Ritter)|-|valign="top"|1:30pm - 3:00pm| Discussion sessions|-|valign="top"|3:00pm - 3:30pm| Coffee & snack break|-|valign="top"|3:30pm - 5:30pm| Discussion sessions|-|valign="top"|5:30pm - 5:45pm| Closing (Matt Ahrens)|- |valign="top"|5:45pm| Beer bash & casual dinner at Delphix office|Developer Summit 2019=== Tuesday, November 19th ==={|border="1" cellpadding="10"|-|valign="top"|9:00am - 9:30am| Breakfast|-|valign="top"|9:30am - 10:00am| Discuss Hackathon ideas & form teams|-|valign="top"|10:00am - 12:30pm|Hackathon|-|valign="top"|12:30pm - 1:30pm|Lunch|-|valign="top"|1:30pm - 5:00pm|Hackathon|-|valign="top"|5:00pm - 5:45pm| Hackathon presentations|-|valign="top"|6:00pm| Optional dinner at [https://www.google.com/maps/preview#!q=Thirsty+Bear+Brewery%2C+Howard+St%2C+San+Francisco%2C+CA&data=!1m4!1m3!1d3309!2d-122.4011122!3d37.7868869!4m15!2m14!1m13!1s0x8085807d0bcfd0df%3A0xdfac6306f9e18402!3m8!1m3!1d13236!2d-122.3926051!3d37.7913156!3m2!1i1261!2i806!4f13.1!4m2!3d37.785502!4d-122.399664 Thirsty Bear]Location is 5 minute walk from Delphix office.|Note that the optional, off-site dinners on Sunday and Tuesday are at your own expense. == Location ==The [http://www.delphix.com/company/about-delphix/ Delphix office] in San Francisco: [https://www.google.com/maps/preview#!q=604+Mission+St.+Suite+400%2C+San+Francisco%2C+CA+94105&data=!1m4!1m3!1d26473!2d-122.4224157!3d37.7867658!4m12!2m11!1m10!1s0x80858062b5adcc23%3A0xd7584093492239e6!3m8!1m3!1d12612!2d-122.3926051!3d37.7913156!3m2!1i1024!2i768!4f13.1 604 Mission Street, 4th floor, San Francisco, CA 94105] Use the keypad to call Delphix and you will be buzzed in. Google maps has pretty good [http://goo.gl/QyUHUj directions]. The easiest way to reach the Delphix SF office is by public transportation. Transportation options, easiest listed first:* BART to the Montgomery St. Station + 3 minute walk* Caltrain to the 4th & King station + 25 minutes walk* Caltrain to the 4th & King station + bus (routes 10, 30, or 45)* drive + park in a lot (costs about $25/day e.g. 41 Tehama Street, 509 Howard Street, or 55 Hawthorne Street) == Discussion topics =='''Note: If you are in the room, you are potentially a speaker.''' This will be done in unconference style, so every attendee should come prepared to present or otherwise lead a discussion or session of up to 45 minutes. The most important things to discuss in this setting are: # Contentious topics that we need to get widespread agreement on# Reports of work in progress that need input from others ==== Proposed topics ====* OpenZFS code repository (Matt)** what code changes are needed to make this a success** process & procedure (code review, testing, committers)* Performance on full or fragmented pools (George, Ned, Alfred)** block allocator improvements* Portable implementation of FMA functionality for Linux & FreeBSD (e.g. for hot spare activation) (Brian)* OpenZFS community organization (Karyn, Jordan, Matt)* Storage tiering (Boris)* How to investigate ZFS performance problems (Adam)* Test suite (John K)* ZFS Channel Programs (Chris, Max G, Matt)* how to examine the ZFS on-disk format with MDB (Max B)* Interactions with virtual memory subsystem (Brian, Jorgen)** dealing with swap on zfs** how the VM system handles large memory allocations* Scalability on large systems, known bottlenecks (Kirill)* How to educate new ZFS developers (Max G)* Disk management - identification, drive spin down/up (John L)* Resumable send/recv (Boris, Max G, Chris) ====Topics that were discussed====* Platform Panel (w/ representatives from illumos, FreeBSD, Linux, and OSX)* Platform-independent code repository (Matt Ahrens)* Storage tiering (Boris Protopopov)* Vendor Panel (all represented companies sharing their work)* Community planning (Karyn Ritter)* Channel programs (Chris Siden & Max Grossman)* Test coverage (John Kennedy)* Performance investigations (Adam Leventhal)* Performance on full/fragmented pools (George Wilson)* Scalability (Kirill Davydychev)* Virtual memory interactions (Brian Behlendorf)* Multi tenancy (Rob Mustacchi)* Examining on-disk format (Max Bruning) == Hackathon ==The goals of the hackathon are:* to get people working with ZFS engineers from outside their normal circles* to start work on some of the [[projects]] we'd like to accomplishTherefore, we should plan to work in small teams of 2-3 engineers. The team members could be pair-programming, splitting up the work into discrete tasks for each person, or advising / implementing. === Hackathon projects ===These projects were worked on at the hackathon: * Testrunner suite running on other platforms: linux, freebsd (+ illumos libzpool)* channel programs: pass instruction stream into kernel* ZFS send progress reporting with "set proc title"* dedicated kmem_cache for dedup table entries* larger (1MB) blocksize * compressed ARC* upstreaming dbuf_user & dbuf state machine (spectra DMU changes)* limit DDT size* onexit for osx === Hackathon ideas ===''Add your idea and your name below. Be prepared to explain your idea at the beginning of the hackathon and enlist/entice others to help with it.''* Userland ioctls: support for send/recv (fd operations over the wire) [[User:mahrens]]* larger (1MB) blocksize: shouldn't be too hard if we ignore performance impact [[User:mahrens]]* dedicated kmem_cache for dedup table entries* stdin / stdout buffer to improve performance of zfs send and receive* ZFS mechanism to override devices' reported sector size (512 vs 4k) (alex @ delphix)* feature flags enhancements for send streams [[User:csiden]]* set owner of all files within a filesystem with a ZFS property [[User:csiden]]* dynamic taskqs for ZIO [[User:ahl]]* towards Oracle Solaris ZFS compatibility [[User:ahl]]* RAIDzN -- full, arbitrary RAID-7 [[User:ahl]]* Testrunner suite running on other platforms (John Kennedy)** or at least identify what is preventing it from running == Registration ==We are expecting everyone will be there to contribute/participate. {|border="1" cellpadding="10"|- !| Name!| Affiliation!| What do you want to talk about?!| Your experience with ZFS!| Travel information!| T-shirt Size!| Food allergies/preferences? (please list)|-| [[User:mahrens | Matt Ahrens]]| Delphix| OpenZFS repo: what interfaces should be supported| Founded ZFS, OpenZFS| Local to SF| M| none|-|[[User:sanjeev | Sanjeev Bagewadi]]| Pluribus Networks| | Was part of the RPE team in Sun which fixed ZFS issues faced by customers| Will be in the Bay Area around that time| M| none|-|-| Ned Bass| LLNL| Common code base, block allocator improvements| ZFS on Linux port, Lustre integration| from East Bay| XL| vegetarian preferred|-| [[User:behlendorf | Brian Behlendorf]]| LLNL| Portable memory management & FMA infrastructure| ZFS on Linux Maintainer| Local to east Bay Area| XL| none|-| Max Bruning| Joyent| examining ZFS on-disk format with MDB| leads ZFS training classes| ?| ?| ?|-| Steven Burgess| Datto Inc.| openZFS as a customer| Working as a developer at a company whose product uses ZOL| from NYC| M| none|-| [[User:kdavyd | Kirill Davydychev]]| Nexenta| Scalability on large systems, major known bottlenecks.| Debugging, performance analysis/tuning| Houston| M| none|-| Justin Gibbs| Spectra Logic| Ref-Links, Resilvering Algorithm, Spares, auto-configuration| DMU performance work| flying in from CO| L| vegetarian|-| [[User:Max | Max Grossman]]| Delphix| Making it easier to get new ZFS developers up to speed (retrospective on the hole birth feature)| 3 months so far, still so much to learn...| Local| L| none|-| Jordan Hubbard| iXsystems| OpenZFS community organization, independent code repo processes| ?| Local| ?| ?|-| Jeremy Jones| Delphix| ?| ZFS bookmarks; debugging techniques| local| ?| ?|-| [[User:jkennedy | John Wren Kennedy]]| Delphix| Uniform test coverage for OpenZFS| Supported ZFS in RPE at Sun/Oracle, and test ZFS in QA at Delphix.| Denver| L| none|-| [[User:jml | John Layman]]| SageCloud| Interested in disk management (spin up/down disks, drive identification, etc.)| Been working with ZFS on Linux for a few months, contributed small bugfix| Boston| XL| none|-| [[User:ahl | Adam Leventhal]]| Delphix| ZFS performance investigations?| RAID-Z2/3, performance work, some small ZFS tasks here and there| local| L| none|-| [[User:delphij | Xin Li]]| iXsystems| ZFS developer for FreeBSD, TrueNAS and FreeNAS products| FreeBSD deputy security officer / release engineering team member| local Bay Area| L| none|-| [[User:lundman | Jorgen Lundman]]| GMO Internet| ZFS, portability| ZFS on OS X| Tokyo| XL| none|-| [[User:maxim | Maxim Martynov]]| | | | | M| none|-| Robert Mustacchi| Joyent|| Misc. debugging| Local to SF| L| None|- | [[User:tonyn | Tony Nguyen]]| Nexenta|| Misc. debugging, import and I/O performance| flying in from Denver| M| None|- | Alfred Perlstein| iXsystems| ZFS performance and debugging| FreeNAS (FreeBSD+ZFS) developer| ?| ?| ?|- | Boris Protopopov| Nexenta| Resumable send/recv, defrag, storage tiering, spare management, doing fancy things with vdevs| ZFS dev lead at Nexenta, ZFS extensions for enterprise class features and performance| Boston| M| None|-| [[User:awreece | Alex Reece]]| Delphix| | small bug fixes| local| M| No marinera|-| [[User:kritter | Karyn Ritter]]| Delphix| Community management (discussion)| | local| M| |-| [[User:rodrigc | Craig Rodrigues]]| iXsystems| ZFS developer for TrueNAS and FreeNAS products| FreeBSD release engineering team member| local| L| none|-| [[User:csiden | Chris Siden]]| Delphix| ZFS Channel Programs, feature flags enhancements for send streams| Worked on feature flags and async destroy| local| L| none|-| Alex Solomatnikov| | | | | M| none|-| [[User:pks | Prashanth Sreenivasa ]]| | I would like to learn more about ZIL, ZFS I/O scheduler & pipeline and impact of using flash-disks instead of conventional hard-disks.| Worked on various PDE (product development engineering) groups including Network Storage, Systems & Solaris kernel. Recently I'm working on ZFS based storage appliances, trying to improve zfs performance on illumos/x86 platform and looking forward to contribute to the open source zfs.| | M| |-| [[User:prakashsurya | Prakash Surya]]| LLNL| | Misc. work on Linux port. Much to learn, still.| Local to east Bay Area| M| none<!--|-| Ilya Usvyatsky| Nexenta| resumable send/recv, defrag| ?| ?| ?| ?-->|-| Rafael Vanoni| Pluribus| I'm not a ZFS developer, but would like to work my way towards that.| I worked on the thread scheduler/dispatcher at Sun/Oracle| ?| M| ?|-| [[User:gwilson | George Wilson]]| Delphix| performance on full & fragmented pools| Implemented lots of SPA & VDEV stuff| flying in from Atlanta ~Monday - Wednesday| L| none|-| Matt Youds| High Availability| | Experience with ZFS configuration at High-Availability.| | L| Fish|-| Sam Zaydel| ?| ?| ?| ?| ?| ?|} == Sponsors ==The event was fully sponsored! Thank you to our sponsors: * Breakfast (2 days) - Sponsored! Thank you [http://www.cloudscaling.com Cloudscaling] and [http://www.ixsystems.com iXsystems]* Lunch (2 days) - Sponsored! Thank you [http://sagecloud.com Sage Cloud], [http://www.spectralogic.com Spectra Logic] and [http://www.freebsdfoundation.org FreeBSD Foundation]* Coffee Breaks (2 days) - Sponsored! Thank you [http://www.ixsystems.com iXsystems]* Beer Bash (day 1) - Sponsored! Thank you [http://www.ddrdrive.com DDRdrive]* T-Shirts - Sponsored! Thank you [http://www.dexlphix.com Delphix]* Video Recordings - Thank you [http://www.joyent.com Joyent]* Cash Donation for expenses - Thank you [http://www.nexenta.com Nexenta] and [http://www.high-availability.com/ High-Availability] What do you get for sponsoring: * Your name/link on this page!* Thank you mention during the opening and closing* Space to put your pop-up banner at the conference venue* An opportunity to be known as a ZFS supporter in a very talented and influential community.
971
edits

Navigation menu