aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2012-01-16Merge branch 'for-chris' of git://repo.or.cz/linux-btrfs-devel into integrationChris Mason
Conflicts: fs/btrfs/volumes.c Signed-off-by: Chris Mason <chris.mason@oracle.com>
2012-01-16Merge branch 'restriper' of git://github.com/idryomov/btrfs-unstable into ↵Chris Mason
integration
2012-01-16Merge branch 'allocation-fixes' into integrationChris Mason
2012-01-16Btrfs: add balance progress reportingIlya Dryomov
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: allow for resuming restriper after it was pausedIlya Dryomov
Recognize BTRFS_BALANCE_RESUME flag passed from userspace. We use the same heuristics used when recovering balance after a crash to try to start where we left off last time. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: allow for canceling restriperIlya Dryomov
Implement an ioctl for canceling restriper. Currently we wait until relocation of the current block group is finished, in future this can be done by triggering a commit. Balance item is deleted and no memory about the interrupted balance is kept. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: allow for pausing restriperIlya Dryomov
Implement an ioctl for pausing restriper. This pauses the relocation, but balance is still considered to be "in progress": balance item is not deleted, other volume operations cannot be started, etc. If paused in the middle of profile changing operation we will continue making allocations with the target profile. Add a hook to close_ctree() to pause restriper and free its data structures on unmount. (It's safe to unmount when restriper is in "paused" state, we will resume with the same parameters on the next mount) Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: add skip_balance mount optionIlya Dryomov
Since restriper kthread starts involuntarily on mount and can suck cpu and memory bandwidth add a mount option to forcefully skip it. The restriper in that case hangs around in paused state and can be resumed from userspace when it's convenient. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: recover balance on mountIlya Dryomov
On mount, if balance item is found, resume balance in a separate kernel thread. Try to be smart to continue roughly where previous balance (or convert) was interrupted. For chunk types that were being converted to some profile we turn on soft convert, in case of a simple balance we turn on usage filter and relocate only less-than-90%-full chunks of that type. These are just heuristics but they help quite a bit, and can be improved in future. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: save balance parameters to diskIlya Dryomov
Introduce a new btree objectid for storing balance item. The reason is to be able to resume restriper after a crash with the same parameters. Balance item has a very high objectid and goes into tree of tree roots. The key for the new item is as follows: [ BTRFS_BALANCE_OBJECTID ; BTRFS_BALANCE_ITEM_KEY ; 0 ] Older kernels simply ignore it so it's safe to mount with an older kernel and then go back to the newer one. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: soft profile changing mode (aka soft convert)Ilya Dryomov
When doing convert from one profile to another if soft mode is on restriper won't touch chunks that already have the profile we are converting to. This is useful if e.g. half of the FS was converted earlier. The soft mode switch is (like every other filter) per-type. This means that we can convert for example meta chunks the "hard" way while converting data chunks selectively with soft switch. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: implement online profile changingIlya Dryomov
Profile changing is done by launching a balance with BTRFS_BALANCE_CONVERT bits set and target fields of respective btrfs_balance_args structs initialized. Profile reducing code in this case will pick restriper's target profile if it's available instead of doing a blind reduce. If target profile is not yet available it goes back to a plain reduce. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: do not reduce profile in do_chunk_alloc()Ilya Dryomov
Every caller of do_chunk_alloc() feeds it the reduced allocation profile, so stop trying to reduce it one more time. Instead check the validity of the passed profile. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: virtual address space subset filterIlya Dryomov
Select chunks which have at least one byte located inside a given [vstart, vend) virtual address space range. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: devid subset filterIlya Dryomov
Select chunks which have at least one byte of at least one stripe located on a device with devid X in a given [pstart,pend) physical address range. This filter only works when devid filter is turned on. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: devid filterIlya Dryomov
Relocate chunks which have at least one stripe located on a device with devid X. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: usage filterIlya Dryomov
Select chunks that are less than X percent full. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: profiles filterIlya Dryomov
Select chunks based on a given profile mask. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: add basic infrastructure for selective balancingIlya Dryomov
This allows to have a separate set of filters for each chunk type (data,meta,sys). The code however is generic and switch on chunk type is only done once. This commit also adds a type filter: it allows to balance for example meta and system chunks w/o touching data ones. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: add basic restriper infrastructureIlya Dryomov
Add basic restriper infrastructure: extended balancing ioctl and all related ioctl data structures, add data structure for tracking restriper's state to fs_info, etc. The semantics of the old balancing ioctl are fully preserved. Explicitly disallow any volume operations when balance is in progress. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: make avail_*_alloc_bits fields dynamicIlya Dryomov
Currently when new chunks are created respective avail_alloc_bits field is updated to reflect profiles of all chunks present in the system. However when chunks are removed profile bits are never cleared. This patch clears profile bit of respective avail_alloc_bits field when the last chunk with that profile is removed. Restriper needs this to properly operate when "downgrading". Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: add BTRFS_AVAIL_ALLOC_BIT_SINGLE bitIlya Dryomov
Right now on-disk BTRFS_BLOCK_GROUP_* profile bits are used for avail_{data,metadata,system}_alloc_bits fields, which gather info about available allocation profiles in the FS. When chunk is created or read from disk, its profile is OR'ed with the corresponding avail_alloc_bits field. Since SINGLE is denoted by 0 in the on-disk format, currently there is no way to tell when such chunks become avaialble. Restriper needs that information, so add a separate bit for SINGLE profile. This bit is going to be in-memory only, it should never be written out to disk, so it's not a disk format change. However to avoid remappings in future, reserve corresponding on-disk bit. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: introduce masks for chunk type and profileIlya Dryomov
Chunk's type and profile are encoded in u64 flags field. Introduce masks to easily access them. Also fix the type of BTRFS_BLOCK_GROUP_* constants, it should be ULL. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-16Btrfs: get rid of *_alloc_profile fieldsIlya Dryomov
{data,metadata,system}_alloc_profile fields have been unused for a long time now. Get rid of them. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-01-11Btrfs: fix possible deadlock when opening a seed deviceLi Zefan
The correct lock order is uuid_mutex -> volume_mutex -> chunk_mutex, but when we mount a filesystem which has backing seed devices, we have this lock chain: open_ctree() lock(chunk_mutex); read_chunk_tree(); read_one_dev(); open_seed_devices(); lock(uuid_mutex); and then we hit a lockdep splat. Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
2012-01-11Btrfs: update global block_rsv when creating a new block groupLi Zefan
A bug was triggered while using seed device: # mkfs.btrfs /dev/loop1 # btrfstune -S 1 /dev/loop1 # mount -o /dev/loop1 /mnt # btrfs dev add /dev/loop2 /mnt btrfs: block rsv returned -28 ------------[ cut here ]------------ WARNING: at fs/btrfs/extent-tree.c:5969 btrfs_alloc_free_block+0x166/0x396 [btrfs]() ... Call Trace: ... [<f7b7c31c>] btrfs_cow_block+0x101/0x147 [btrfs] [<f7b7eaa6>] btrfs_search_slot+0x1b8/0x55f [btrfs] [<f7b7f844>] btrfs_insert_empty_items+0x42/0x7f [btrfs] [<f7b7f8c1>] btrfs_insert_item+0x40/0x7e [btrfs] [<f7b8ac02>] btrfs_make_block_group+0x243/0x2aa [btrfs] [<f7bb3f53>] __btrfs_alloc_chunk+0x672/0x70e [btrfs] [<f7bb41ff>] init_first_rw_device+0x77/0x13c [btrfs] [<f7bb5a62>] btrfs_init_new_device+0x664/0x9fd [btrfs] [<f7bbb65a>] btrfs_ioctl+0x694/0xdbe [btrfs] [<c04f55f7>] do_vfs_ioctl+0x496/0x4cc [<c04f5660>] sys_ioctl+0x33/0x4f [<c07b9edf>] sysenter_do_call+0x12/0x38 ---[ end trace 906adac595facc7d ]--- Since seed device is readonly, there's no usable space in the filesystem. Afterwards we add a sprout device to it, and the kernel creates a METADATA block group and a SYSTEM block group where comes free space we can reserve, but we still get revervation failure because the global block_rsv hasn't been updated accordingly. Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
2012-01-11Btrfs: rewrite btrfs_trim_block_group()Li Zefan
There are various bugs in block group trimming: - It may trim from offset smaller than user-specified offset. - It may trim beyond user-specified range. - It may leak free space for extents smaller than specified minlen. - It may truncate the last trimmed extent thus leak free space. - With mixed extents+bitmaps, some extents may not be trimmed. - With mixed extents+bitmaps, some bitmaps may not be trimmed (even none will be trimmed). Even for those trimmed, not all the free space in the bitmaps will be trimmed. I rewrite btrfs_trim_block_group() and break it into two functions. One is to trim extents only, and the other is to trim bitmaps only. Before patching: # fstrim -v /mnt/ /mnt/: 1496465408 bytes were trimmed After patching: # fstrim -v /mnt/ /mnt/: 2193768448 bytes were trimmed And this matches the total free space: # btrfs fi df /mnt Data: total=3.58GB, used=1.79GB System, DUP: total=8.00MB, used=4.00KB System: total=4.00MB, used=0.00 Metadata, DUP: total=205.12MB, used=97.14MB Metadata: total=8.00MB, used=0.00 Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
2012-01-11Btrfs: simplfy calculation of stripe length for discard operationLi Zefan
For btrfs raid, while discarding a range of space, we'll need to know the start offset and length to discard for each device, and it's done in btrfs_map_block(). However the calculation is a bit complex for raid0 and raid10, so I reimplement it based on a fact that: dev1 dev2 dev3 (raid0) ----------------------------------- s0 s3 s6 s1 s4 s7 s2 s5 Each device has (total_stripes / nr_dev) stripes, or plus one. Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
2012-01-11Btrfs: don't pre-allocate btrfs bioLi Zefan
We pre-allocate a btrfs bio with fixed size, and then may re-allocate memory if we find stripes are bigger than the fixed size. But this pre-allocation is not necessary. Also we don't have to calcuate the stripe number twice. Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
2012-01-11Btrfs: don't pass a trans handle unnecessarily in volumes.cLi Zefan
Some functions never use the transaction handle passed to them. Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
2012-01-11Btrfs: reserve metadata space in btrfs_ioctl_setflags()Li Zefan
Check and reserve space for btrfs_update_inode(). Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
2012-01-11Btrfs: remove BUG_ON()s in btrfs_ioctl_setflags()Li Zefan
We can recover from errors and return -errno to user space. Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
2012-01-11Btrfs: check the return value of io_ctl_init()Li Zefan
It can return -ENOMEM. Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
2012-01-11Btrfs: avoid possible NULL deref in io_ctl_drop_pages()Li Zefan
If we run into some failure path in io_ctl_prepare_pages(), io_ctl->pages[] array may have some NULL pointers. Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
2012-01-11Btrfs: add pinned extents to on-disk free space cache correctlyLi Zefan
I got this while running xfstests: [24256.836098] block group 317849600 has an wrong amount of free space [24256.836100] btrfs: failed to load free space cache for block group 317849600 We should clamp the extent returned by find_first_extent_bit(), so the start of the extent won't smaller than the start of the block group. Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
2012-01-11Merge branch 'for-linus' of ↵Li Zefan
git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs into for-linus
2012-01-07Btrfs: revamp clustered allocation logicAlexandre Oliva
Parameterize clusters on minimum total size, minimum chunk size and minimum contiguous size for at least one chunk, without limits on cluster, window or gap sizes. Don't tolerate any fragmentation for SSD_SPREAD; accept it for metadata, but try to keep data dense. Signed-off-by: Alexandre Oliva <oliva@lsd.ic.unicamp.br> Signed-off-by: Chris Mason <chris.mason@oracle.com>
2012-01-07Btrfs: don't set up allocation result twiceAlexandre Oliva
We store the allocation start and length twice in ins, once right after the other, but with intervening calls that may prevent the duplicate from being optimized out by the compiler. Remove one of the assignments. Signed-off-by: Alexandre Oliva <oliva@lsd.ic.unicamp.br> Signed-off-by: Chris Mason <chris.mason@oracle.com>
2012-01-06Btrfs: test free space only for unclustered allocationAlexandre Oliva
Since the clustered allocation may be taking extents from a different block group, there's no point in spin-locking and testing the current block group free space before attempting to allocate space from a cluster, even more so when we might refrain from even trying the cluster in the current block group because, after the cluster was set up, not enough free space remained. Furthermore, cluster creation attempts fail fast when the block group doesn't have enough free space, so the test was completely superfluous. I've move the free space test past the cluster allocation attempt, where it is more useful, and arranged for a cluster in the current block group to be released before trying an unclustered allocation, when we reach the LOOP_NO_EMPTY_SIZE stage, so that the free space in the cluster stands a chance of being combined with additional free space in the block group so as to succeed in the allocation attempt. Signed-off-by: Alexandre Oliva <oliva@lsd.ic.unicamp.br> Signed-off-by: Chris Mason <chris.mason@oracle.com>
2012-01-06Btrfs: use bigger metadata chunks on bigger filesystemsChris Mason
The 256MB chunk is a little small on a huge FS. This scales up the chunk size. Signed-off-by: Chris Mason <chris.mason@oracle.com>
2012-01-06Btrfs: lower the bar for chunk allocationChris Mason
The chunk allocation code has tried to keep a pretty tight lid on creating new metadata chunks. This is partially because in the past the reservation code didn't give us an accurate idea of how much space was being used. The new code is much more accurate, so we're able to get rid of some of these checks. Signed-off-by: Chris Mason <chris.mason@oracle.com>
2012-01-06Btrfs: run chunk allocations while we do delayed refsChris Mason
Btrfs tries to batch extent allocation tree changes to improve performance and reduce metadata trashing. But it doesn't allocate new metadata chunks while it is doing allocations for the extent allocation tree. This commit changes the delayed refence code to do chunk allocations if we're getting low on room. It prevents crashes and improves performance. Signed-off-by: Chris Mason <chris.mason@oracle.com>
2012-01-04Linux 3.2Linus Torvalds
2012-01-04Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/netLinus Torvalds
* git://git.kernel.org/pub/scm/linux/kernel/git/davem/net: fix CAN MAINTAINERS SCM tree type mwifiex: fix crash during simultaneous scan and connect b43: fix regression in PIO case ath9k: Fix kernel panic in AR2427 in AP mode CAN MAINTAINERS update net: fsl: fec: fix build for mx23-only kernel sch_qfq: fix overflow in qfq_update_start() Revert "Bluetooth: Increase HCI reset timeout in hci_dev_do_close"
2012-01-04minixfs: misplaced checks lead to dentry leakAl Viro
bitmap size sanity checks should be done *before* allocating ->s_root; there their cleanup on failure would be correct. As it is, we do iput() on root inode, but leak the root dentry... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk> Acked-by: Josh Boyer <jwboyer@redhat.com> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-01-04ptrace: ensure JOBCTL_STOP_SIGMASK is not zero after detachOleg Nesterov
This is the temporary simple fix for 3.2, we need more changes in this area. 1. do_signal_stop() assumes that the running untraced thread in the stopped thread group is not possible. This was our goal but it is not yet achieved: a stopped-but-resumed tracee can clone the running thread which can initiate another group-stop. Remove WARN_ON_ONCE(!current->ptrace). 2. A new thread always starts with ->jobctl = 0. If it is auto-attached and this group is stopped, __ptrace_unlink() sets JOBCTL_STOP_PENDING but JOBCTL_STOP_SIGMASK part is zero, this triggers WANR_ON(!signr) in do_jobctl_trap() if another debugger attaches. Change __ptrace_unlink() to set the artificial SIGSTOP for report. Alternatively we could change ptrace_init_task() to copy signr from current, but this means we can copy it for no reason and hide the possible similar problems. Acked-by: Tejun Heo <tj@kernel.org> Cc: <stable@kernel.org> [3.1] Signed-off-by: Oleg Nesterov <oleg@redhat.com> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-01-04ptrace: partially fix the do_wait(WEXITED) vs EXIT_DEAD->EXIT_ZOMBIE raceOleg Nesterov
Test-case: int main(void) { int pid, status; pid = fork(); if (!pid) { for (;;) { if (!fork()) return 0; if (waitpid(-1, &status, 0) < 0) { printf("ERR!! wait: %m\n"); return 0; } } } assert(ptrace(PTRACE_ATTACH, pid, 0,0) == 0); assert(waitpid(-1, NULL, 0) == pid); assert(ptrace(PTRACE_SETOPTIONS, pid, 0, PTRACE_O_TRACEFORK) == 0); do { ptrace(PTRACE_CONT, pid, 0, 0); pid = waitpid(-1, NULL, 0); } while (pid > 0); return 1; } It fails because ->real_parent sees its child in EXIT_DEAD state while the tracer is going to change the state back to EXIT_ZOMBIE in wait_task_zombie(). The offending commit is 823b018e which moved the EXIT_DEAD check, but in fact we should not blame it. The original code was not correct as well because it didn't take ptrace_reparented() into account and because we can't really trust ->ptrace. This patch adds the additional check to close this particular race but it doesn't solve the whole problem. We simply can't rely on ->ptrace in this case, it can be cleared if the tracer is multithreaded by the exiting ->parent. I think we should kill EXIT_DEAD altogether, we should always remove the soon-to-be-reaped child from ->children or at least we should never do the DEAD->ZOMBIE transition. But this is too complex for 3.2. Reported-and-tested-by: Denys Vlasenko <vda.linux@googlemail.com> Tested-by: Lukasz Michalik <lmi@ift.uni.wroc.pl> Acked-by: Tejun Heo <tj@kernel.org> Cc: <stable@kernel.org> [3.0+] Signed-off-by: Oleg Nesterov <oleg@redhat.com> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-01-04Merge git://git.samba.org/sfrench/cifs-2.6Linus Torvalds
* git://git.samba.org/sfrench/cifs-2.6: [CIFS] default ntlmv2 for cifs mount delayed to 3.3 cifs: fix bad buffer length check in coalesce_t2
2012-01-04Merge branch 'master' of ↵John W. Linville
git://git.kernel.org/pub/scm/linux/kernel/git/linville/wireless into for-davem
2012-01-04Revert "rtc: Expire alarms after the time is set."Linus Torvalds
This reverts commit 93b2ec0128c431148b216b8f7337c1a52131ef03. The call to "schedule_work()" in rtc_initialize_alarm() happens too early, and can cause oopses at bootup Neil Brown explains why we do it: "If you set an alarm in the future, then shutdown and boot again after that time, then you will end up with a timer_queue node which is in the past. When this happens the queue gets stuck. That entry-in-the-past won't get removed until and interrupt happens and an interrupt won't happen because the RTC only triggers an interrupt when the alarm is "now". So you'll find that e.g. "hwclock" will always tell you that 'select' timed out. So we force the interrupt work to happen at the start just in case." and has a patch that convert it to do things in-process rather than with the worker thread, but right now it's too late to play around with this, so we just revert the patch that caused problems for now. Reported-by: Sander Eikelenboom <linux@eikelenboom.it> Requested-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Requested-by: John Stultz <john.stultz@linaro.org> Cc: Neil Brown <neilb@suse.de> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>