Jump to content
NAKIVO Introduces Agent-Based Data Protection for Proxmox VE ×
NAKIVO Community Forum

10.4 on Ubuntu 18.04 introduces serious NFS repository issues


Gavino

Recommended Posts

I have a fresh install of Ubuntu Server 18.04 with nothing but the following done to it before installing Nakivo "NAKIVO_Backup_Replication_v10.4.0.56979_Installer-TRIAL.sh"

sudo apt update
sudo apt install nfs-common
sudo apt install cifs-utils
sudo apt install open-iscsi
sudo apt install ntfs-3g

I installed Nakivo with flags " --eula-accept -f -C" since I am using NFS instead of onboard repository.

You can see in 10.4 that multiple fields are missing.  Where is the data storage policy ("Forever incremental" etc)? Where is the deduplication option? I don't want compression in Nakivo as my TrueNAS storage does LZ4 compression already. Even though Data Size Reduction is set to Disabled, it's still telling me that fast compression is set (even though I toggled that to Disabled, before setting Data Size Reduction to Disabled).

If I experiment and turn the Data Size Reduction" to Enabled, I am then able to change compression to Disabled and have that stick, but still - deduplication remains off, and that matters greatly to me. Still no idea what my data storage policy is. I uninstalled 10.4 and installed 10.3 on the same box, and that is behaving correctly. Compare the screenshots.

10.4 is a hot mess for NFS right now and I simply cannot use it. You broke login wallpapers in 10.3, which was annoying, but this is simply horrible. Where are your testing procedures?

Support bundle from 10.4 has been sent. I am running 10.3 now so you'll have to replicate in your lab, which should be easy to do.

 

 

 

 

Nakivo 10.3 NFS repository creation.png

Nakivo 10.3 NFS repository view.png

Nakivo 10.4 NFS repository creation.png

Nakivo 10.4 NFS repository view.png

Nakivo 10.4 Data size reduction Enabled.png

  • Like 1
Link to comment
Share on other sites

OK I worked out that "Store backups in separate files" is the exact same meaning as "no deduplication". If I am wrong - then please let me know.

Nakivo should have just kept it simple by keeping the "deduplication" wording and not confuse people with another option that means the same thing. It's a bit crazy to look at a repository after it's been created, and see "Deduplication" and "Store backups in separate files" as completely separate line items (which are far apart). Both refer to the same thing. I bet there is a product designer pointing his finger at a collegue right now going "see, I told you that customers would not like this!". And they would be right.

It's pretty strange that for this feature, the new docs are now saying "Enabling this option is highly recommended to ensure higher reliability and performance" WHAT?! Dedup was one of the main reasons I bought Nakivo! https://www.nakivo.com/vmware-backup/#benefits - this talks up dedup as a benefit and cost saving. Nowhere in the sales pitch do you say that it's highly recommended not to use it!!

It's interesting to note also that deduplication and immutable backup respository are incompatible with each other, as I found out from emailing tech support.

Some feedback on documentation. Nakivo should put 10.3 or 10.4 (or whatever the version is) at the top of each documentation page. That would be very handy. I thought I was reading the latest version when I posted above, as the Release Notes page mentioned 10.4. I was like... ok this looks like I'm in the right place for the latest info as it lists 10.4.  https://helpcenter.nakivo.com/display/RN/Release+Notes  I didn't realise until after contacting tech support, that it was the older documentation. Also, the images zoom on the 10.3 docs but not 10.4.  I prefer zoom, as sometimes the info is unreadable until it's zoomed in. Please fix this in the 10.4 docs. Thanks.

An issue I found here:

https://helpcenter.nakivo.com/User-Guide/Content/Deployment/System-Requirements/Feature-Requirements.htm#Ransomware

------ quote

For Local Folder type of Backup Repository, the following conditions must be met:

  • The Backup Repository data storage type must be set to Incremental with full backups.

----- end quote

Whoever wrote that maybe doesn't realise that "incremental with full backups" doesn't exist as an option any more in 10.4, like it did in 10.3. I have been told in an email that "...there is the option "store backups in separate files". When this option is selected, the repository is "Incremental with Full Backup" and Immutable is supported."

It would be handy if you explained that in your documentation. The line above sould probably say

The Backup Repository data storage type must be set to "store backups in separate files" (i.e. deduplication switched off). This implicitly sets the data storage policy to Incremental with full backups.

  • Like 1
Link to comment
Share on other sites

Thanks for the link Mario.  For my use I still really like global deduplication, for the space/cost savings, and indeed it might be necessary given my storage space requirements.

I've recently upgraded from an older Dell R710 6-disk setup to a newer Dell R730 8-disk setup, and switched from SATA to SAS drives. I definitely recommend SAS over SATA - especially for VM Flash Boot and Screenshot Verification. It's worth the extra $$ IMHO. The firmware quality is usally better too, as is the SMART reporting.

Previously with SATA I found it too slow to have "Enforce explicit file system sync" on. The SATA bus is only half-duplex, and only 6Gbps, vs the full-duplex SAS3 12Gbps. Now with my new setup I have exlicit filesystem sync turned on and still get about 1.2-1.5Gbps speeds, which I'm happy with. It was terrible on the old system so had it off there.

With my old setup (without filesystem sync) I did have one corruption on a single VM which has about 2TB of data - a Flash VM Boot would always come up with disk errors and would take forever to try and fix those on boot with checkdisk, which meant Screenshot Verification wouldn't work for that one, but my other 8 VMs were fine.

My main repo is now a high quality TrueNAS Core 12.0-U5 ZFS setup with enterprise SAS drives and ECC buffered RDIMMs, and the ZFS dataset shared via NFS to my Nakivo Director on Ubuntu 18.04 respects the explicit filesystem sync setting set by Nakivo. I have everything on power conditioning UPS, so hoping I don't have any further repo corruptions. I put a 2TB NVMe drive as L2ARC for my ZFS pool, so hoping that gives a nice speed boost to the VM Flash Boot and Screenshot Verification, even after months of backups (and dedup complexity).

Another thing I did was turn off compression for Nakivo, as my NFS datastore uses LZ4 ZFS compression natively, so avoiding double-compression issues. Now that Nakivo only has to worry about dedup, and not dedup + compression, I am hoping that my new setup will be super reliable, even with global dedup on.

I really think global dedup is still fine to use from a reliablity standpoint, but you just have to manage the setup with more care  (quality ECC RAM, quality UPS, high quaity (SAS) drives, high quality drive controller, filesystem sync, optimal compression setings, keep the BIOS and controller+drive firmwares up to date etc). Anyway I'll see how I go.

  • Like 1
Link to comment
Share on other sites

Hello, @Gavinoand @Mario!

Thank you a lot for your feedback. I am very sorry to hear about your unpleasant experience.

We have already escalated this information to our Product Team. Currently, it is being discussed by the Product Team.

I hope that we will solve this issue as soon as possible. Thank you once again for helping us improve the solution!

Link to comment
Share on other sites

There were mistakes on my end - I didn't realise I was reading out-of-date documentation, as the documentation only seems have version number on the front page, and I arrived at the info via direct link of a Google search.  Even with the most current documentation, I think many long-term users will struggle to understand fully what is going on with these changes. Thanks for following up.

  • Like 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...