Welcome to Centmin Mod Community
Become a Member

MariaDB MariaDB 10.2.x might see MyRocks Storage Engine added from Facebook

Discussion in 'MariaDB & General MySQL news & discussions' started by eva2000, Sep 22, 2016.

  1. eva2000

    eva2000 Administrator Staff Member

    54,584
    12,224
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,793
    Local Time:
    4:13 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Just came across this Facebook article on how they created MyRocks storage engine as an alternative to InnoDB for better disk space usage, lower write amplification and faster replication in their Oracle MySQL 5.6 forked version. Which has contributing features to their opensource fork WebScaleSQL MySQL server. Doesn't seem MyRocks made it into WebScaleSQL yet though ? MariaDB 10.1 also has backported a few features from WebScaleSQL MySQL server too :)

    Article: MyRocks: A space- and write-optimized MySQL database | Engineering Blog | Facebook Code | Facebook

    Seems this MariaDB Jira entry has MyRocks storage engine being potentially added in MariaDB 10.2 branch (which is currently in early Alpha release) [MDEV-9658] FR: Add RocksDB as a storage engine to MariaDB Server - JIRA

    @Xon might like the more efficient compression and on disk space size :)


     
  2. Revenge

    Revenge Active Member

    469
    93
    28
    Feb 21, 2016
    Portugal
    Ratings:
    +354
    Local Time:
    6:13 PM
    1.9.x
    10.1.x
  3. eva2000

    eva2000 Administrator Staff Member

    54,584
    12,224
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,793
    Local Time:
    4:13 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    sweet can't wait for MariaDB 10.2 to land with MyRocks storage engine :)

     
  4. eva2000

    eva2000 Administrator Staff Member

    54,584
    12,224
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,793
    Local Time:
    4:13 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    another from same blog Small Datum: MyRocks and InnoDB with large objects and compression

     
  5. eva2000

    eva2000 Administrator Staff Member

    54,584
    12,224
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,793
    Local Time:
    4:13 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
  6. eva2000

    eva2000 Administrator Staff Member

    54,584
    12,224
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,793
    Local Time:
    4:13 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    bit of a limitation Facebook MySQL 5.6 can only support one storage engine at a time either InnoDB or MyRocks Migrating from InnoDB to RocksDB · facebook/mysql-5.6 Wiki · GitHub

     
  7. Xon

    Xon Active Member

    173
    61
    28
    Nov 16, 2015
    Ratings:
    +229
    Local Time:
    2:13 AM
    1.15.x
    MariaDB 10.3.x
    This looks massively interesting. XenForo post/conversation/ip tables are basically nearly purely appending only, with limited updates, and obviously selects. And get really large.

    I just wonder if this requires converting all tables from InnoDB to MyRocks once it lands in MariaDB. I hope it is just a matter of an "alter table engine=MyRocks" than the current migration process.
     
  8. Xon

    Xon Active Member

    173
    61
    28
    Nov 16, 2015
    Ratings:
    +229
    Local Time:
    2:13 AM
    1.15.x
    MariaDB 10.3.x
    Digging through the github issues; I think there are blockers regarding performance of multi-row transaction which update large number of rows.

    But without testing I can't tell how baddly this compares to a large XenForo install using compressed tables for some moderation actions :p
     
  9. eva2000

    eva2000 Administrator Staff Member

    54,584
    12,224
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,793
    Local Time:
    4:13 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    indeed hope MariaDB's implementation is not InnoDB or MyRocks only.
    Indeed, with MariaDB 10.2 in alpha we still have time to see how this pans out. Subscribed and watching the jira entry though [MDEV-9658] FR: Add RocksDB as a storage engine to MariaDB Server - JIRA :D
     
    Last edited: Sep 22, 2016
  10. Revenge

    Revenge Active Member

    469
    93
    28
    Feb 21, 2016
    Portugal
    Ratings:
    +354
    Local Time:
    6:13 PM
    1.9.x
    10.1.x
    Many changes about this in MariaDB Jira today.
     
  11. eva2000

    eva2000 Administrator Staff Member

    54,584
    12,224
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,793
    Local Time:
    4:13 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
  12. Revenge

    Revenge Active Member

    469
    93
    28
    Feb 21, 2016
    Portugal
    Ratings:
    +354
    Local Time:
    6:13 PM
    1.9.x
    10.1.x
  13. eva2000

    eva2000 Administrator Staff Member

    54,584
    12,224
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,793
    Local Time:
    4:13 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    indeed official confirmation of MyRocks in MariaDB 10.2 https://twitter.com/markcallaghan/status/784020503799304192 :)

    just waiting :D
    Code (Text):
    mysqladmin ver
    mysqladmin  Ver 9.1 Distrib 10.2.2-MariaDB, for Linux on x86_64
    Copyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.
    
    Server version          10.2.2-MariaDB
    Protocol version        10
    Connection              Localhost via UNIX socket
    UNIX socket             /var/lib/mysql/mysql.sock
    Uptime:                 4 days 11 hours 58 min 19 sec
    
    Threads: 2  Questions: 1295159  Slow queries: 0  Opens: 48  Flush tables: 1  Open tables: 18  Queries per second avg: 3.332
     
  14. Xon

    Xon Active Member

    173
    61
    28
    Nov 16, 2015
    Ratings:
    +229
    Local Time:
    2:13 AM
    1.15.x
    MariaDB 10.3.x
    Looks awesome.

    But I want replication to work with this awesome new table format :p
     
  15. eva2000

    eva2000 Administrator Staff Member

    54,584
    12,224
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,793
    Local Time:
    4:13 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Indeed hoping MariaDB folks can make that happen :)
     
  16. Revenge

    Revenge Active Member

    469
    93
    28
    Feb 21, 2016
    Portugal
    Ratings:
    +354
    Local Time:
    6:13 PM
    1.9.x
    10.1.x
  17. eva2000

    eva2000 Administrator Staff Member

    54,584
    12,224
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,793
    Local Time:
    4:13 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
  18. Revenge

    Revenge Active Member

    469
    93
    28
    Feb 21, 2016
    Portugal
    Ratings:
    +354
    Local Time:
    6:13 PM
    1.9.x
    10.1.x
    MariaDB now have a date for the stable 10.2 version. Is set to 22 of December with version 10.2.5
     
  19. eva2000

    eva2000 Administrator Staff Member

    54,584
    12,224
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,793
    Local Time:
    4:13 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Sweet :D
     
  20. eva2000

    eva2000 Administrator Staff Member

    54,584
    12,224
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,793
    Local Time:
    4:13 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+