[OmniOS-discuss] zfs and small files

[Chavdar Ivanov]

From: Chavdar Ivanov [mailto:ci4 at outlook.com] 
Sent: 13 November 2013 16:15
To: 'Peter Tribble'; 'Tobias Oetiker'
Cc: omnios-discuss at lists.omniti.com
Subject: RE: [OmniOS-discuss] zfs and small files

 

 

 

From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On
Behalf Of Peter Tribble
Sent: 13 November 2013 15:36
To: Tobias Oetiker
Cc: omnios-discuss at lists.omniti.com <mailto:omnios-discuss at lists.omniti.com>

Subject: Re: [OmniOS-discuss] zfs and small files

 

On Wed, Nov 13, 2013 at 2:46 PM, Tobias Oetiker <tobi at oetiker.ch
<mailto:tobi at oetiker.ch> > wrote:

Hi Peter,

Today Peter Tribble wrote:

> On Wed, Nov 13, 2013 at 1:59 PM, Tobias Oetiker <tobi at oetiker.ch
<mailto:tobi at oetiker.ch> > wrote:
>
> > We have this customer who loves to put tons of files into single
> > directories.  The other day he found that removing a directory with
> > 7k files from raidz2 pool (with ssd log device) took about 10
> > minutes ...
> >
> > is this to be expected, or should this be faster on a properly
> > configured system ?
> >
>
> That's a tiny number of files. Ought to be essentially instant.
> I routinely have up to a million files in a directory. That can take
> a few minutes to delete, but I don't have any SSDs at all on
> those systems.
>
> This isn't being done over NFS, is it? That can dramatically slow
> operations like this, even with a log.

nope it is local ...

are you running this on raidz2 ?

 

Yes. On old X4540s. Deletion of a directory with 7k files in it

takes about 0.3s

[ci] 

It will take about 0.3s IIF you have snapshots of the ZFS. It will take
looooong time if this is the only snapshot of the system. 

 

Chavdar

[ci] 

(my usual "method" of deleting many files on a system without a snapshot is
to create one, delete them as required, then remove the snapshot at a
convenient time - at the end of the day or so. 

CI

 

 

-- 
-Peter Tribble
http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omniosce.org/ml-archive/attachments/20131113/24eb811c/attachment.html>