-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEZH8oZUiU471FcZm+ONu9yGCSaT4FAl3K9vkACgkQONu9yGCS
aT6fag//aZGV3gVyTHo9ZFFSs1ZO6AW9H/Qyfi+w1Q+pfvMfxvOroFs0Fojk1xoa
VQ9QDpa9O9dvX1GcvCaxwToVR8zte+wWMviDOz5Tae5puFQuT3dkjleMiU/5Gu+E
wWwV7dLoyJ4WMIvRCuxs7djh9NYQDczkOCLZtA1z9IF2SkG12pOJ5yg2ZOOUnovJ
lvSo7phGYv3X/tZHUvWvg6XG8DaAzWYgo4nQ0Ccv0Q3IWiqAlJFZwWH+VDqgzdPS
HKwKKqLqjTo2mfmDbScduVtIton6U9wipXrnOkqVPNxbrMS1OpmNreYfJYRXfCfZ
0WD/pEqa4t9dAkRVXhal91CXjJ0pfl5VPo3T3EXEDSCBb/rvId45dMpKxLO4kLmm
rF7w0NdvcSV9Zo07m6SL8S+ADrgBNts6g2ZopFRCm7HRXxxGXlA0smLR8mUrsMxw
49Lvcr0MtVu8/LgrnPEjyvT1A615Yad5f3n7oJSKsQrm/32abrTb47ZE98U7rUnU
9NEiBrLe83dyq3HTPbPspx7yIAjiCkknQdmCylL+lLIlRaIXeas81D+AO+zO3Wyu
DBjJRy8N+PotluK66PxlKSaJSrkH2YbQsbYUpifU9KZ8JfR2CRmEzmb4cyWgpVOH
2clUBuQg6uEoDK2QsweIdKvHlCO6vHKIE4Gh71NSvS8vxMfg+tA=
=WHB8
-----END PGP SIGNATURE-----
Merge 4.9.201 into android-4.9-q
Changes in 4.9.201
CDC-NCM: handle incomplete transfer of MTU
ipv4: Fix table id reference in fib_sync_down_addr
net: fix data-race in neigh_event_send()
net: usb: qmi_wwan: add support for DW5821e with eSIM support
NFC: fdp: fix incorrect free object
nfc: netlink: fix double device reference drop
NFC: st21nfca: fix double free
qede: fix NULL pointer deref in __qede_remove()
ALSA: timer: Fix incorrectly assigned timer instance
ALSA: bebob: fix to detect configured source of sampling clock for Focusrite Saffire Pro i/o series
ALSA: hda/ca0132 - Fix possible workqueue stall
mm: thp: handle page cache THP correctly in PageTransCompoundMap
mm, vmstat: hide /proc/pagetypeinfo from normal users
dump_stack: avoid the livelock of the dump_lock
perf tools: Fix time sorting
drm/radeon: fix si_enable_smc_cac() failed issue
ceph: fix use-after-free in __ceph_remove_cap()
iio: imu: adis16480: make sure provided frequency is positive
netfilter: nf_tables: Align nft_expr private data to 64-bit
netfilter: ipset: Fix an error code in ip_set_sockfn_get()
can: usb_8dev: fix use-after-free on disconnect
can: c_can: c_can_poll(): only read status register after status IRQ
can: peak_usb: fix a potential out-of-sync while decoding packets
can: gs_usb: gs_can_open(): prevent memory leak
can: peak_usb: fix slab info leak
configfs: Fix bool initialization/comparison
configfs: stash the data we need into configfs_buffer at open time
configfs_register_group() shouldn't be (and isn't) called in rmdirable parts
configfs: new object reprsenting tree fragments
configfs: provide exclusion between IO and removals
configfs: fix a deadlock in configfs_symlink()
usbip: stub_rx: fix static checker warning on unnecessary checks
usbip: Fix vhci_urb_enqueue() URB null transfer buffer error path
usbip: fix possibility of dereference by NULLL pointer in vhci_hcd.c
drivers: usb: usbip: Add missing break statement to switch
PCI: tegra: Enable Relaxed Ordering only for Tegra20 & Tegra30
dmaengine: xilinx_dma: Fix control reg update in vdma_channel_set_config
HID: intel-ish-hid: fix wrong error handling in ishtp_cl_alloc_tx_ring()
scsi: qla2xxx: fixup incorrect usage of host_byte
scsi: lpfc: Honor module parameter lpfc_use_adisc
ipvs: move old_secure_tcp into struct netns_ipvs
bonding: fix unexpected IFF_BONDING bit unset
usb: fsl: Check memory resource before releasing it
usb: gadget: udc: atmel: Fix interrupt storm in FIFO mode.
usb: gadget: composite: Fix possible double free memory bug
usb: gadget: configfs: fix concurrent issue between composite APIs
usb: dwc3: remove the call trace of USBx_GFLADJ
perf/x86/amd/ibs: Fix reading of the IBS OpData register and thus precise RIP validity
perf/x86/amd/ibs: Handle erratum #420 only on the affected CPU family (10h)
USB: Skip endpoints with 0 maxpacket length
RDMA/iw_cxgb4: Avoid freeing skb twice in arp failure case
scsi: qla2xxx: stop timer in shutdown path
fjes: Handle workqueue allocation failure
net: hisilicon: Fix "Trying to free already-free IRQ"
NFSv4: Don't allow a cached open with a revoked delegation
net: ethernet: arc: add the missed clk_disable_unprepare
igb: Fix constant media auto sense switching when no cable is connected
e1000: fix memory leaks
x86/apic: Move pending interrupt check code into it's own function
x86/apic: Drop logical_smp_processor_id() inline
x86/apic/32: Avoid bogus LDR warnings
can: flexcan: disable completely the ECC mechanism
mm/filemap.c: don't initiate writeback if mapping has no dirty pages
cgroup,writeback: don't switch wbs immediately on dead wbs if the memcg is dead
net: prevent load/store tearing on sk->sk_stamp
drm/i915: kick out cmd_parser specific structs from i915_drv.h
drm/i915: cleanup use of INSTR_CLIENT_MASK
drm/i915: return EACCES for check_cmd() failures
drm/i915: don't whitelist oacontrol in cmd parser
drm/i915: Use the precomputed value for whether to enable command parsing
drm/i915/cmdparser: Limit clflush to active cachelines
drm/i915/gtt: Add read only pages to gen8_pte_encode
drm/i915/gtt: Read-only pages for insert_entries on bdw+
drm/i915/gtt: Disable read-only support under GVT
drm/i915: Prevent writing into a read-only object via a GGTT mmap
drm/i915/cmdparser: Check reg_table_count before derefencing.
drm/i915/cmdparser: Do not check past the cmd length.
drm/i915: Silence smatch for cmdparser
drm/i915: Move engine->needs_cmd_parser to engine->flags
drm/i915: Rename gen7 cmdparser tables
drm/i915: Disable Secure Batches for gen6+
drm/i915: Remove Master tables from cmdparser
drm/i915: Add support for mandatory cmdparsing
drm/i915: Support ro ppgtt mapped cmdparser shadow buffers
drm/i915: Allow parsing of unsized batches
drm/i915: Add gen9 BCS cmdparsing
drm/i915/cmdparser: Use explicit goto for error paths
drm/i915/cmdparser: Add support for backward jumps
drm/i915/cmdparser: Ignore Length operands during command matching
drm/i915: Lower RM timeout to avoid DSI hard hangs
drm/i915/gen8+: Add RC6 CTX corruption WA
drm/i915/cmdparser: Fix jump whitelist clearing
Linux 4.9.201
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
commit 65de03e251382306a4575b1779c57c87889eee49 upstream.
cgroup writeback tries to refresh the associated wb immediately if the
current wb is dead. This is to avoid keeping issuing IOs on the stale
wb after memcg - blkcg association has changed (ie. when blkcg got
disabled / enabled higher up in the hierarchy).
Unfortunately, the logic gets triggered spuriously on inodes which are
associated with dead cgroups. When the logic is triggered on dead
cgroups, the attempt fails only after doing quite a bit of work
allocating and initializing a new wb.
While c3aab9a0bd91 ("mm/filemap.c: don't initiate writeback if mapping
has no dirty pages") alleviated the issue significantly as it now only
triggers when the inode has dirty pages. However, the condition can
still be triggered before the inode is switched to a different cgroup
and the logic simply doesn't make sense.
Skip the immediate switching if the associated memcg is dying.
This is a simplified version of the following two patches:
* https://lore.kernel.org/linux-mm/20190513183053.GA73423@dennisz-mbp/
* http://lkml.kernel.org/r/156355839560.2063.5265687291430814589.stgit@buzz
Cc: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Fixes: e8a7abf5a5 ("writeback: disassociate inodes from dying bdi_writebacks")
Acked-by: Dennis Zhou <dennis@kernel.org>
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEZH8oZUiU471FcZm+ONu9yGCSaT4FAl1GilkACgkQONu9yGCS
aT6ibw//QcoRk+6jjWIzW2R83tvcSpDREIYKF5xDPULqmaUKv7qcbGqdCT4OI0Vf
CySao6jvmN+1V+5OlQHHw1qmithHF/Y4yEVi+eL0CcMgB6qA2tQxeA0ffFu0Fzxe
oKrAIDANAJ2FKymqbIzTJU8AChNuy+Rc+C60L9O0ZhBHykLYvO7/VepTxdO2aMGs
a9tdZyXnOAr/ZgKx+uN1F8Rx2ZorNfFmP2rDxEZ/B8pVrXsOjMAcxWRsZBv+w9mc
zWaMPEL1vIR/kG35L18l2EFwZ++uIGvfGR2HxhNRUlTqN4m/3trATIc0eRn5PyAC
qBlVdcwUeJKavBqK3cgHvK9CzWQdSMxNefk9A0H66ZGpfSuNCiFjw54AXEiRzx3t
OvzUvDjfa36jsKW7yiYXdLbEa52gT14UDzmSIzlAYQLBplEadHikJ0FxCFzAWpFt
C13gG1e4G0ZKEHH8wZMuRdanHbN87c/0Sm4rgTLMwCO6I1PeILcUwIq9El9M4RVL
MmHSXLKduaHbPJWx+Qopl6NxjqTe/VR8paT6q1QnU00/4kP15aIVE08vZSiGSNP+
Gp6xbv12skAx1m1zP7K82oGdwnCVCJUqlC9wafcjsaxcoCVBWvIgPkyMWAzUeFzc
Ub2yIS9iulUeYOxPXBZKWCqgwpl7kovGztf4gwX2Kuy50mXSZQg=
=ozfF
-----END PGP SIGNATURE-----
Merge 4.9.187 into android-4.9-q
Changes in 4.9.187
MIPS: ath79: fix ar933x uart parity mode
MIPS: fix build on non-linux hosts
arm64/efi: Mark __efistub_stext_offset as an absolute symbol explicitly
dmaengine: imx-sdma: fix use-after-free on probe error path
ath10k: Do not send probe response template for mesh
ath9k: Check for errors when reading SREV register
ath6kl: add some bounds checking
ath: DFS JP domain W56 fixed pulse type 3 RADAR detection
batman-adv: fix for leaked TVLV handler.
media: dvb: usb: fix use after free in dvb_usb_device_exit
crypto: talitos - fix skcipher failure due to wrong output IV
media: marvell-ccic: fix DMA s/g desc number calculation
media: vpss: fix a potential NULL pointer dereference
media: media_device_enum_links32: clean a reserved field
net: stmmac: dwmac1000: Clear unused address entries
net: stmmac: dwmac4/5: Clear unused address entries
signal/pid_namespace: Fix reboot_pid_ns to use send_sig not force_sig
af_key: fix leaks in key_pol_get_resp and dump_sp.
xfrm: Fix xfrm sel prefix length validation
media: mc-device.c: don't memset __user pointer contents
media: staging: media: davinci_vpfe: - Fix for memory leak if decoder initialization fails.
net: phy: Check against net_device being NULL
crypto: talitos - properly handle split ICV.
crypto: talitos - Align SEC1 accesses to 32 bits boundaries.
tua6100: Avoid build warnings.
locking/lockdep: Fix merging of hlocks with non-zero references
media: wl128x: Fix some error handling in fm_v4l2_init_video_device()
cpupower : frequency-set -r option misses the last cpu in related cpu list
net: fec: Do not use netdev messages too early
net: axienet: Fix race condition causing TX hang
s390/qdio: handle PENDING state for QEBSM devices
perf cs-etm: Properly set the value of 'old' and 'head' in snapshot mode
perf test 6: Fix missing kvm module load for s390
gpio: omap: fix lack of irqstatus_raw0 for OMAP4
gpio: omap: ensure irq is enabled before wakeup
regmap: fix bulk writes on paged registers
bpf: silence warning messages in core
rcu: Force inlining of rcu_read_lock()
blkcg, writeback: dead memcgs shouldn't contribute to writeback ownership arbitration
xfrm: fix sa selector validation
perf evsel: Make perf_evsel__name() accept a NULL argument
vhost_net: disable zerocopy by default
ipoib: correcly show a VF hardware address
EDAC/sysfs: Fix memory leak when creating a csrow object
ipsec: select crypto ciphers for xfrm_algo
media: i2c: fix warning same module names
ntp: Limit TAI-UTC offset
timer_list: Guard procfs specific code
acpi/arm64: ignore 5.1 FADTs that are reported as 5.0
media: coda: fix mpeg2 sequence number handling
media: coda: increment sequence offset for the last returned frame
mt7601u: do not schedule rx_tasklet when the device has been disconnected
x86/build: Add 'set -e' to mkcapflags.sh to delete broken capflags.c
mt7601u: fix possible memory leak when the device is disconnected
ath10k: fix PCIE device wake up failed
perf tools: Increase MAX_NR_CPUS and MAX_CACHES
libata: don't request sense data on !ZAC ATA devices
clocksource/drivers/exynos_mct: Increase priority over ARM arch timer
rslib: Fix decoding of shortened codes
rslib: Fix handling of of caller provided syndrome
ixgbe: Check DDM existence in transceiver before access
crypto: asymmetric_keys - select CRYPTO_HASH where needed
EDAC: Fix global-out-of-bounds write when setting edac_mc_poll_msec
bcache: check c->gc_thread by IS_ERR_OR_NULL in cache_set_flush()
iwlwifi: mvm: Drop large non sta frames
net: usb: asix: init MAC address buffers
gpiolib: Fix references to gpiod_[gs]et_*value_cansleep() variants
Bluetooth: hci_bcsp: Fix memory leak in rx_skb
Bluetooth: 6lowpan: search for destination address in all peers
Bluetooth: Check state in l2cap_disconnect_rsp
Bluetooth: validate BLE connection interval updates
gtp: fix Illegal context switch in RCU read-side critical section.
gtp: fix use-after-free in gtp_newlink()
xen: let alloc_xenballooned_pages() fail if not enough memory free
scsi: NCR5380: Reduce goto statements in NCR5380_select()
scsi: NCR5380: Always re-enable reselection interrupt
scsi: mac_scsi: Increase PIO/PDMA transfer length threshold
crypto: ghash - fix unaligned memory access in ghash_setkey()
crypto: arm64/sha1-ce - correct digest for empty data in finup
crypto: arm64/sha2-ce - correct digest for empty data in finup
crypto: chacha20poly1305 - fix atomic sleep when using async algorithm
crypto: crypto4xx - fix a potential double free in ppc4xx_trng_probe
Input: gtco - bounds check collection indent level
regulator: s2mps11: Fix buck7 and buck8 wrong voltages
arm64: tegra: Update Jetson TX1 GPU regulator timings
iwlwifi: pcie: don't service an interrupt that was masked
tracing/snapshot: Resize spare buffer if size changed
NFSv4: Handle the special Linux file open access mode
lib/scatterlist: Fix mapping iterator when sg->offset is greater than PAGE_SIZE
ALSA: seq: Break too long mutex context in the write loop
ALSA: hda/realtek: apply ALC891 headset fixup to one Dell machine
media: v4l2: Test type instead of cfg->type in v4l2_ctrl_new_custom()
media: coda: Remove unbalanced and unneeded mutex unlock
KVM: x86/vPMU: refine kvm_pmu err msg when event creation failed
arm64: tegra: Fix AGIC register range
fs/proc/proc_sysctl.c: fix the default values of i_uid/i_gid on /proc/sys inodes.
drm/nouveau/i2c: Enable i2c pads & busses during preinit
padata: use smp_mb in padata_reorder to avoid orphaned padata jobs
9p/virtio: Add cleanup path in p9_virtio_init
PCI: Do not poll for PME if the device is in D3cold
Btrfs: add missing inode version, ctime and mtime updates when punching hole
libnvdimm/pfn: fix fsdax-mode namespace info-block zero-fields
take floppy compat ioctls to sodding floppy.c
floppy: fix div-by-zero in setup_format_params
floppy: fix out-of-bounds read in next_valid_format
floppy: fix invalid pointer dereference in drive_name
floppy: fix out-of-bounds read in copy_buffer
coda: pass the host file in vma->vm_file on mmap
gpu: ipu-v3: ipu-ic: Fix saturation bit offset in TPMEM
crypto: ccp - Validate the the error value used to index error messages
PCI: hv: Delete the device earlier from hbus->children for hot-remove
PCI: hv: Fix a use-after-free bug in hv_eject_device_work()
crypto: caam - limit output IV to CBC to work around CTR mode DMA issue
um: Allow building and running on older hosts
um: Fix FP register size for XSTATE/XSAVE
parisc: Ensure userspace privilege for ptraced processes in regset functions
parisc: Fix kernel panic due invalid values in IAOQ0 or IAOQ1
powerpc/32s: fix suspend/resume when IBATs 4-7 are used
powerpc/watchpoint: Restore NV GPRs while returning from exception
eCryptfs: fix a couple type promotion bugs
intel_th: msu: Fix single mode with disabled IOMMU
Bluetooth: Add SMP workaround Microsoft Surface Precision Mouse bug
usb: Handle USB3 remote wakeup for LPM enabled devices correctly
dm bufio: fix deadlock with loop device
compiler.h, kasan: Avoid duplicating __read_once_size_nocheck()
compiler.h: Add read_word_at_a_time() function.
lib/strscpy: Shut up KASAN false-positives in strscpy()
ext4: allow directory holes
bnx2x: Prevent load reordering in tx completion processing
bnx2x: Prevent ptp_task to be rescheduled indefinitely
caif-hsi: fix possible deadlock in cfhsi_exit_module()
igmp: fix memory leak in igmpv3_del_delrec()
ipv4: don't set IPv6 only flags to IPv4 addresses
net: bcmgenet: use promisc for unsupported filters
net: dsa: mv88e6xxx: wait after reset deactivation
net: neigh: fix multiple neigh timer scheduling
net: openvswitch: fix csum updates for MPLS actions
nfc: fix potential illegal memory access
rxrpc: Fix send on a connected, but unbound socket
sky2: Disable MSI on ASUS P6T
vrf: make sure skb->data contains ip header to make routing
macsec: fix use-after-free of skb during RX
macsec: fix checksumming after decryption
netrom: fix a memory leak in nr_rx_frame()
netrom: hold sock when setting skb->destructor
bonding: validate ip header before check IPPROTO_IGMP
tcp: Reset bytes_acked and bytes_received when disconnecting
net: bridge: mcast: fix stale nsrcs pointer in igmp3/mld2 report handling
net: bridge: mcast: fix stale ipv6 hdr pointer when handling v6 query
net: bridge: stp: don't cache eth dest pointer before skb pull
perf/x86/amd/uncore: Rename 'L2' to 'LLC'
perf/x86/amd/uncore: Get correct number of cores sharing last level cache
perf/events/amd/uncore: Fix amd_uncore_llc ID to use pre-defined cpu_llc_id
NFSv4: Fix open create exclusive when the server reboots
nfsd: increase DRC cache limit
nfsd: give out fewer session slots as limit approaches
nfsd: fix performance-limiting session calculation
nfsd: Fix overflow causing non-working mounts on 1 TB machines
drm/panel: simple: Fix panel_simple_dsi_probe
usb: core: hub: Disable hub-initiated U1/U2
tty: max310x: Fix invalid baudrate divisors calculator
pinctrl: rockchip: fix leaked of_node references
tty: serial: cpm_uart - fix init when SMC is relocated
drm/bridge: tc358767: read display_props in get_modes()
drm/bridge: sii902x: pixel clock unit is 10kHz instead of 1kHz
memstick: Fix error cleanup path of memstick_init
tty/serial: digicolor: Fix digicolor-usart already registered warning
tty: serial: msm_serial: avoid system lockup condition
serial: 8250: Fix TX interrupt handling condition
drm/virtio: Add memory barriers for capset cache.
phy: renesas: rcar-gen2: Fix memory leak at error paths
drm/rockchip: Properly adjust to a true clock in adjusted_mode
tty: serial_core: Set port active bit in uart_port_activate
usb: gadget: Zero ffs_io_data
powerpc/pci/of: Fix OF flags parsing for 64bit BARs
PCI: sysfs: Ignore lockdep for remove attribute
kbuild: Add -Werror=unknown-warning-option to CLANG_FLAGS
PCI: xilinx-nwl: Fix Multi MSI data programming
iio: iio-utils: Fix possible incorrect mask calculation
recordmcount: Fix spurious mcount entries on powerpc
mfd: core: Set fwnode for created devices
mfd: arizona: Fix undefined behavior
mfd: hi655x-pmic: Fix missing return value check for devm_regmap_init_mmio_clk
um: Silence lockdep complaint about mmap_sem
powerpc/4xx/uic: clear pending interrupt after irq type/pol change
RDMA/i40iw: Set queue pair state when being queried
serial: sh-sci: Terminate TX DMA during buffer flushing
serial: sh-sci: Fix TX DMA buffer flushing and workqueue races
kallsyms: exclude kasan local symbols on s390
perf test mmap-thread-lookup: Initialize variable to suppress memory sanitizer warning
RDMA/rxe: Fill in wc byte_len with IB_WC_RECV_RDMA_WITH_IMM
powerpc/boot: add {get, put}_unaligned_be32 to xz_config.h
f2fs: avoid out-of-range memory access
mailbox: handle failed named mailbox channel request
powerpc/eeh: Handle hugepages in ioremap space
sh: prevent warnings when using iounmap
mm/kmemleak.c: fix check for softirq context
9p: pass the correct prototype to read_cache_page
mm/mmu_notifier: use hlist_add_head_rcu()
locking/lockdep: Fix lock used or unused stats error
locking/lockdep: Hide unused 'class' variable
usb: wusbcore: fix unbalanced get/put cluster_id
usb: pci-quirks: Correct AMD PLL quirk detection
x86/sysfb_efi: Add quirks for some devices with swapped width and height
x86/speculation/mds: Apply more accurate check on hypervisor platform
hpet: Fix division by zero in hpet_time_div()
ALSA: line6: Fix wrong altsetting for LINE6_PODHD500_1
ALSA: hda - Add a conexant codec entry to let mute led work
powerpc/tm: Fix oops on sigreturn on systems without TM
access: avoid the RCU grace period for the temporary subjective credentials
ipv6: check sk sk_type and protocol early in ip_mroute_set/getsockopt
tcp: reset sk_send_head in tcp_write_queue_purge
arm64: dts: marvell: Fix A37xx UART0 register size
i2c: qup: fixed releasing dma without flush operation completion
arm64: compat: Provide definition for COMPAT_SIGMINSTKSZ
ISDN: hfcsusb: checking idx of ep configuration
media: au0828: fix null dereference in error path
media: cpia2_usb: first wake up, then free in disconnect
media: radio-raremono: change devm_k*alloc to k*alloc
Bluetooth: hci_uart: check for missing tty operations
sched/fair: Don't free p->numa_faults with concurrent readers
drivers/pps/pps.c: clear offset flags in PPS_SETPARAMS ioctl
ceph: hold i_ceph_lock when removing caps for freeing inode
Linux 4.9.187
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
[ Upstream commit 6631142229005e1b1c311a09efe9fb3cfdac8559 ]
wbc_account_io() collects information on cgroup ownership of writeback
pages to determine which cgroup should own the inode. Pages can stay
associated with dead memcgs but we want to avoid attributing IOs to
dead blkcgs as much as possible as the association is likely to be
stale. However, currently, pages associated with dead memcgs
contribute to the accounting delaying and/or confusing the
arbitration.
Fix it by ignoring pages associated with dead memcgs.
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Jan Kara <jack@suse.cz>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Sasha Levin <sashal@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEZH8oZUiU471FcZm+ONu9yGCSaT4FAlzkK/4ACgkQONu9yGCS
aT7MpBAAwnw0HPr67VKF/PD18KNz5e8suu9OD8lUI7z5+CVA62bAe4cx4FQNOudZ
gR9iS2Q57k0b6pKIX1sOITkAuJCHIH7J9vL8PDIu+NeJFJiz82vg/k/3dq/QjL64
1Z6jjcDu/i4yNF/Jv+eAMbs1faRDOdMM4uLHqpwqdoOhp6aKVkqmPe1gMSXpCF4U
KTaypH/3f/0cMCWVSv75qPBK4eiiW5WgoS3CWtOcZAaxqpQpPdo3zKgzjxgfYCIR
7U0LpTSp9EEFIKtOYNvAJEUb2WZRKqSXHhPaWFmfYsTL6Zl5S4V7rbQaepmrKHpV
uzHSsPt6cXrzCnfPu0kcXzPDrQrW5zDg70ndYhuxR0mChcwazopRIVtDRHP6DXso
g/F1tKORhz5DPYuAkz7GCdyFQNXqipz6ybx54Z3JC7C+kXO6mc0GzFmihMjrlWCg
oIK0Va6WL7f1aYsakv1ngI2v7NqVq/EylOFHkqMZMxlrtUOWKbznz5CHg9D5oxO4
P/cZBkOhmsvKqtV5wtKDAISXmAIQMQabgKlPa7jPayjBk9b/4EEXp2DAJJ3lnNPQ
KfK+W6iChZho+YIyd4KWGpqu88bmDUssR11tw2xAJlnFKv2e1+tQQaCUjdp1meUM
ewbOuBEuElmPWMmh6TACzxa1pF6cPW72C1JJ1xYoUx/yihUo6n0=
=js4a
-----END PGP SIGNATURE-----
Merge 4.9.178 into android-4.9-q
Changes in 4.9.178
net: core: another layer of lists, around PF_MEMALLOC skb handling
locking/rwsem: Prevent decrement of reader count before increment
PCI: hv: Fix a memory leak in hv_eject_device_work()
x86/speculation/mds: Revert CPU buffer clear on double fault exit
x86/speculation/mds: Improve CPU buffer clear documentation
objtool: Fix function fallthrough detection
ARM: exynos: Fix a leaked reference by adding missing of_node_put
power: supply: axp288_charger: Fix unchecked return value
arm64: compat: Reduce address limit
arm64: Clear OSDLR_EL1 on CPU boot
sched/x86: Save [ER]FLAGS on context switch
crypto: chacha20poly1305 - set cra_name correctly
crypto: vmx - fix copy-paste error in CTR mode
crypto: crct10dif-generic - fix use via crypto_shash_digest()
crypto: x86/crct10dif-pcl - fix use via crypto_shash_digest()
ALSA: usb-audio: Fix a memory leak bug
ALSA: hda/hdmi - Read the pin sense from register when repolling
ALSA: hda/hdmi - Consider eld_valid when reporting jack event
ALSA: hda/realtek - EAPD turn on later
ASoC: max98090: Fix restore of DAPM Muxes
ASoC: RT5677-SPI: Disable 16Bit SPI Transfers
mm/mincore.c: make mincore() more conservative
ocfs2: fix ocfs2 read inode data panic in ocfs2_iget
mfd: da9063: Fix OTP control register names to match datasheets for DA9063/63L
mfd: max77620: Fix swapped FPS_PERIOD_MAX_US values
tty/vt: fix write/write race in ioctl(KDSKBSENT) handler
jbd2: check superblock mapped prior to committing
ext4: actually request zeroing of inode table after grow
ext4: fix ext4_show_options for file systems w/o journal
Btrfs: do not start a transaction at iterate_extent_inodes()
bcache: fix a race between cache register and cacheset unregister
bcache: never set KEY_PTRS of journal key to 0 in journal_reclaim()
ipmi:ssif: compare block number correctly for multi-part return messages
crypto: gcm - Fix error return code in crypto_gcm_create_common()
crypto: gcm - fix incompatibility between "gcm" and "gcm_base"
crypto: salsa20 - don't access already-freed walk.iv
crypto: arm/aes-neonbs - don't access already-freed walk.iv
fib_rules: fix error in backport of e9919a24d302 ("fib_rules: return 0...")
writeback: synchronize sync(2) against cgroup writeback membership switches
fs/writeback.c: use rcu_barrier() to wait for inflight wb switches going into workqueue when umount
ext4: zero out the unused memory region in the extent tree block
ext4: fix data corruption caused by overlapping unaligned and aligned IO
ALSA: hda/realtek - Fix for Lenovo B50-70 inverted internal microphone bug
KVM: x86: Skip EFER vs. guest CPUID checks for host-initiated writes
Linux 4.9.178
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
commit ec084de929e419e51bcdafaafe567d9e7d0273b7 upstream.
synchronize_rcu() didn't wait for call_rcu() callbacks, so inode wb
switch may not go to the workqueue after synchronize_rcu(). Thus
previous scheduled switches was not finished even flushing the
workqueue, which will cause a NULL pointer dereferenced followed below.
VFS: Busy inodes after unmount of vdd. Self-destruct in 5 seconds. Have a nice day...
BUG: unable to handle kernel NULL pointer dereference at 0000000000000278
evict+0xb3/0x180
iput+0x1b0/0x230
inode_switch_wbs_work_fn+0x3c0/0x6a0
worker_thread+0x4e/0x490
? process_one_work+0x410/0x410
kthread+0xe6/0x100
ret_from_fork+0x39/0x50
Replace the synchronize_rcu() call with a rcu_barrier() to wait for all
pending callbacks to finish. And inc isw_nr_in_flight after call_rcu()
in inode_switch_wbs() to make more sense.
Link: http://lkml.kernel.org/r/20190429024108.54150-1-jiufei.xue@linux.alibaba.com
Signed-off-by: Jiufei Xue <jiufei.xue@linux.alibaba.com>
Acked-by: Tejun Heo <tj@kernel.org>
Suggested-by: Tejun Heo <tj@kernel.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
commit 7fc5854f8c6efae9e7624970ab49a1eac2faefb1 upstream.
sync_inodes_sb() can race against cgwb (cgroup writeback) membership
switches and fail to writeback some inodes. For example, if an inode
switches to another wb while sync_inodes_sb() is in progress, the new
wb might not be visible to bdi_split_work_to_wbs() at all or the inode
might jump from a wb which hasn't issued writebacks yet to one which
already has.
This patch adds backing_dev_info->wb_switch_rwsem to synchronize cgwb
switch path against sync_inodes_sb() so that sync_inodes_sb() is
guaranteed to see all the target wbs and inodes can't jump wbs to
escape syncing.
v2: Fixed misplaced rwsem init. Spotted by Jiufei.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-by: Jiufei Xue <xuejiufei@gmail.com>
Link: http://lkml.kernel.org/r/dc694ae2-f07f-61e1-7097-7c8411cee12d@gmail.com
Acked-by: Jan Kara <jack@suse.cz>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEZH8oZUiU471FcZm+ONu9yGCSaT4FAlr75w0ACgkQONu9yGCS
aT7CFQ/+J5ciLm7pMN7mQadoQsssK8CRTBVd1VKgyrMGvh9+EC8ZBcQd7sHLcmuY
lJ0V4g41D2WvESjTBHa73aCTAmxtfA9DGzhlXoNxGkm9syZt3E3DI254axvVm1aU
PGNC/A3C64nuAgSz2BkGtEfg/DbDihy6XqB/J/monLoL07hPtJ2wn3C8jv9J+TGw
jidRnkMcmcp9TsIytzOgbNEfKWOrf+SvFHaadNQKa28ApKN/bfTZ+ncbdXOfcHGF
6pKPko4O7UCfTG+IG6k9+fQLJaTDhspnl0b7tCKBGbboRSb4EuexM57NQDElKt8u
c4MDoQz2DeOl9RJFQnCiQ9bLixwce7PJEawNrlFUcFmJQprBSsn/4mlPlT4feZ3n
CEyGMKk0a/RKTAVcBTAqEkPTRkrrkI/z4FJNEyedANpeaQzl0pCPBo1RPvWJ7Vim
PDSlDgoxjo8H6vOssHC+8ncg2qFVt/tPT3t/ZjV0tDU68II9xSn7hK0MXvmwx9hM
qC8Ryq0KuuHmoogodaDAc/4H/JpNX4xSAMLKtZX+uRt9lyDK3mGMS18EAl7lLXQr
S8sXf9ohvUzR/4v9YXDRS+fbcnre5hR5BfOhlomiZra+Gz7GdG6e6C5hHs4RPPc+
BUz1fMnE/ZVv5FSJqrEe4XF1+1t3NyEqXajR2q2sFfO/4ocJWiY=
=pwA9
-----END PGP SIGNATURE-----
Merge 4.9.100 into android-4.9
Changes in 4.9.100
ipvs: fix rtnl_lock lockups caused by start_sync_thread
IB/device: Convert ib-comp-wq to be CPU-bound
kcm: Call strp_stop before strp_done in kcm_attach
crypto: af_alg - fix possible uninit-value in alg_bind()
netlink: fix uninit-value in netlink_sendmsg
net: fix rtnh_ok()
net: initialize skb->peeked when cloning
net: fix uninit-value in __hw_addr_add_ex()
dccp: initialize ireq->ir_mark
soreuseport: initialise timewait reuseport field
perf: Remove superfluous allocation error check
tcp: fix TCP_REPAIR_QUEUE bound checking
bdi: Fix oops in wb_workfn()
KVM: PPC: Book3S HV: Fix trap number return from __kvmppc_vcore_entry
f2fs: fix a dead loop in f2fs_fiemap()
arm64: Add work around for Arm Cortex-A55 Erratum 1024718
gpioib: do not free unrequested descriptors
gpio: fix aspeed_gpio unmask irq
gpio: fix error path in lineevent_create
rfkill: gpio: fix memory leak in probe error path
libata: Apply NOLPM quirk for SanDisk SD7UB3Q*G1001 SSDs
tracing: Fix regex_match_front() to not over compare the test string
can: kvaser_usb: Increase correct stats counter in kvaser_usb_rx_can_msg()
drm/vc4: Fix scaling of uni-planar formats
drm/i915: Fix drm:intel_enable_lvds ERROR message in kernel log
net: atm: Fix potential Spectre v1
atm: zatm: Fix potential Spectre v1
Revert "Bluetooth: btusb: Fix quirk for Atheros 1525/QCA6174"
thermal: exynos: Reading temperature makes sense only when TMU is turned on
thermal: exynos: Propagate error value from tmu_read()
tracing/uprobe_event: Fix strncpy corner case
perf/x86: Fix possible Spectre-v1 indexing for hw_perf_event cache_*
perf/x86/cstate: Fix possible Spectre-v1 indexing for pkg_msr
perf/x86/msr: Fix possible Spectre-v1 indexing in the MSR driver
perf/core: Fix possible Spectre-v1 indexing for ->aux_pages[]
perf/x86: Fix possible Spectre-v1 indexing for x86_pmu::event_map()
Linux 4.9.100
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
commit b8b784958eccbf8f51ebeee65282ca3fd59ea391 upstream.
Syzbot has reported that it can hit a NULL pointer dereference in
wb_workfn() due to wb->bdi->dev being NULL. This indicates that
wb_workfn() was called for an already unregistered bdi which should not
happen as wb_shutdown() called from bdi_unregister() should make sure
all pending writeback works are completed before bdi is unregistered.
Except that wb_workfn() itself can requeue the work with:
mod_delayed_work(bdi_wq, &wb->dwork, 0);
and if this happens while wb_shutdown() is waiting in:
flush_delayed_work(&wb->dwork);
the dwork can get executed after wb_shutdown() has finished and
bdi_unregister() has cleared wb->bdi->dev.
Make wb_workfn() use wakeup_wb() for requeueing the work which takes all
the necessary precautions against racing with bdi unregistration.
CC: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
CC: Tejun Heo <tj@kernel.org>
Fixes: 839a8e8660
Reported-by: syzbot <syzbot+9873874c735f2892e7e9@syzkaller.appspotmail.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEZH8oZUiU471FcZm+ONu9yGCSaT4FAlre3foACgkQONu9yGCS
aT4ygw/7BxK85/7my26TCcF3NlNfw8x9B1AuEHcjQdUFmSFip9fbolC1VVFQyzWV
1Z6sK5LKrFU/234A4LVW+6oFVnUi8VvX5RnWNqRLj/xpPyv/L5PsG4A7VwQLZDCj
yn6OrIkQ2czv3oio9R8bz1DFxrPsxoCdrMzV1xR/tbWQJscSRSomBSNstZ63lGQP
FleS//hG21092lM0EDPQRfc9/xnwJFMKJoCwBrf6SUR7CK2ZdYp+w+HoeP5BSA49
BCDdMU6wi63gTdfGFJa76VcKuFxRRic3F9VVJwOuYPCfCpgNz7GiUaGXirNCRSxD
F0VveFMx9lOrxMCyDDEoj3lRaZ4FrhwFOF+Q0RK4/p8XH8Pr9emvt4beOeYNk3ev
dFvCI/zA3zmDj8ppB+ZnKq9GTRSuNZ5sw4q7AyFu6fm0mSMV84ZRHgCrVbJkis3i
VDnu6V+Nd9ZYP6rR6AegtQwFM3OReVVJG4ZpB17PZOebLdM+n4DsmqDCgvhFboVP
WowtlR13GIn6ER7hiHfOlwv3CtTUspRrAZi73F8aEA7gdoYzGc8/9Q1fA738WiI4
46utL18qyfjzwbwmI9dtFx1SLkrobSJ+X88PLquLYNUleoxbLx/2u4+v+gm0P4xo
yQGPjIv+kms+rbwPR3ogMKqC2GrYYJs4uHBEPJboJWzYVoPAaEs=
=yoJG
-----END PGP SIGNATURE-----
Merge 4.9.96 into android-4.9
Changes in 4.9.96
tty: make n_tty_read() always abort if hangup is in progress
ubifs: Check ubifs_wbuf_sync() return code
ubi: fastmap: Don't flush fastmap work on detach
ubi: Fix error for write access
ubi: Reject MLC NAND
fs/reiserfs/journal.c: add missing resierfs_warning() arg
resource: fix integer overflow at reallocation
ipc/shm: fix use-after-free of shm file via remap_file_pages()
mm, slab: reschedule cache_reap() on the same CPU
usb: musb: gadget: misplaced out of bounds check
usb: gadget: udc: core: update usb_ep_queue() documentation
ARM: dts: at91: at91sam9g25: fix mux-mask pinctrl property
ARM: dts: exynos: Fix IOMMU support for GScaler devices on Exynos5250
ARM: dts: at91: sama5d4: fix pinctrl compatible string
spi: Fix scatterlist elements size in spi_map_buf
xen-netfront: Fix hang on device removal
regmap: Fix reversed bounds check in regmap_raw_write()
ACPI / video: Add quirk to force acpi-video backlight on Samsung 670Z5E
ACPI / hotplug / PCI: Check presence of slot itself in get_slot_status()
USB: gadget: f_midi: fixing a possible double-free in f_midi
USB:fix USB3 devices behind USB3 hubs not resuming at hibernate thaw
usb: dwc3: pci: Properly cleanup resource
smb3: Fix root directory when server returns inode number of zero
HID: i2c-hid: fix size check and type usage
powerpc/powernv: Handle unknown OPAL errors in opal_nvram_write()
powerpc/64: Fix smp_wmb barrier definition use use lwsync consistently
powerpc/powernv: define a standard delay for OPAL_BUSY type retry loops
powerpc/powernv: Fix OPAL NVRAM driver OPAL_BUSY loops
HID: Fix hid_report_len usage
HID: core: Fix size as type u32
ASoC: ssm2602: Replace reg_default_raw with reg_default
thunderbolt: Resume control channel after hibernation image is created
irqchip/gic: Take lock when updating irq type
random: use a tighter cap in credit_entropy_bits_safe()
jbd2: if the journal is aborted then don't allow update of the log tail
ext4: don't update checksum of new initialized bitmaps
ext4: protect i_disksize update by i_data_sem in direct write path
ext4: fail ext4_iget for root directory if unallocated
RDMA/ucma: Don't allow setting RDMA_OPTION_IB_PATH without an RDMA device
RDMA/rxe: Fix an out-of-bounds read
ALSA: pcm: Fix UAF at PCM release via PCM timer access
IB/srp: Fix srp_abort()
IB/srp: Fix completion vector assignment algorithm
dmaengine: at_xdmac: fix rare residue corruption
libnvdimm, namespace: use a safe lookup for dimm device name
nfit, address-range-scrub: fix scrub in-progress reporting
um: Compile with modern headers
um: Use POSIX ucontext_t instead of struct ucontext
iommu/vt-d: Fix a potential memory leak
mmc: jz4740: Fix race condition in IRQ mask update
clk: mvebu: armada-38x: add support for 1866MHz variants
clk: mvebu: armada-38x: add support for missing clocks
clk: fix false-positive Wmaybe-uninitialized warning
clk: bcm2835: De-assert/assert PLL reset signal when appropriate
pwm: rcar: Fix a condition to prevent mismatch value setting to duty
thermal: imx: Fix race condition in imx_thermal_probe()
dt-bindings: clock: mediatek: add binding for fixed-factor clock axisel_d4
watchdog: f71808e_wdt: Fix WD_EN register read
vfio/pci: Virtualize Maximum Read Request Size
ALSA: pcm: Use ERESTARTSYS instead of EINTR in OSS emulation
ALSA: pcm: Avoid potential races between OSS ioctls and read/write
ALSA: pcm: Return -EBUSY for OSS ioctls changing busy streams
ALSA: pcm: Fix mutex unbalance in OSS emulation ioctls
ALSA: pcm: Fix endless loop for XRUN recovery in OSS emulation
ext4: don't allow r/w mounts if metadata blocks overlap the superblock
drm/amdgpu: Add an ATPX quirk for hybrid laptop
drm/amdgpu: Fix always_valid bos multiple LRU insertions.
drm/amdgpu: Fix PCIe lane width calculation
drm/rockchip: Clear all interrupts before requesting the IRQ
drm/radeon: Fix PCIe lane width calculation
ALSA: line6: Use correct endpoint type for midi output
ALSA: rawmidi: Fix missing input substream checks in compat ioctls
ALSA: hda - New VIA controller suppor no-snoop path
random: fix crng_ready() test
random: crng_reseed() should lock the crng instance that it is modifying
random: add new ioctl RNDRESEEDCRNG
HID: hidraw: Fix crash on HIDIOCGFEATURE with a destroyed device
MIPS: uaccess: Add micromips clobbers to bzero invocation
MIPS: memset.S: EVA & fault support for small_memset
MIPS: memset.S: Fix return of __clear_user from Lpartial_fixup
MIPS: memset.S: Fix clobber of v1 in last_fixup
powerpc/eeh: Fix enabling bridge MMIO windows
powerpc/lib: Fix off-by-one in alternate feature patching
udf: Fix leak of UTF-16 surrogates into encoded strings
jffs2_kill_sb(): deal with failed allocations
hypfs_kill_super(): deal with failed allocations
orangefs_kill_sb(): deal with allocation failures
rpc_pipefs: fix double-dput()
Don't leak MNT_INTERNAL away from internal mounts
autofs: mount point create should honour passed in mode
mm/filemap.c: fix NULL pointer in page_cache_tree_insert()
fanotify: fix logic of events on child
writeback: safer lock nesting
block/mq: fix potential deadlock during cpu hotplug
Linux 4.9.96
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
commit 2e898e4c0a3897ccd434adac5abb8330194f527b upstream.
lock_page_memcg()/unlock_page_memcg() use spin_lock_irqsave/restore() if
the page's memcg is undergoing move accounting, which occurs when a
process leaves its memcg for a new one that has
memory.move_charge_at_immigrate set.
unlocked_inode_to_wb_begin,end() use spin_lock_irq/spin_unlock_irq() if
the given inode is switching writeback domains. Switches occur when
enough writes are issued from a new domain.
This existing pattern is thus suspicious:
lock_page_memcg(page);
unlocked_inode_to_wb_begin(inode, &locked);
...
unlocked_inode_to_wb_end(inode, locked);
unlock_page_memcg(page);
If both inode switch and process memcg migration are both in-flight then
unlocked_inode_to_wb_end() will unconditionally enable interrupts while
still holding the lock_page_memcg() irq spinlock. This suggests the
possibility of deadlock if an interrupt occurs before unlock_page_memcg().
truncate
__cancel_dirty_page
lock_page_memcg
unlocked_inode_to_wb_begin
unlocked_inode_to_wb_end
<interrupts mistakenly enabled>
<interrupt>
end_page_writeback
test_clear_page_writeback
lock_page_memcg
<deadlock>
unlock_page_memcg
Due to configuration limitations this deadlock is not currently possible
because we don't mix cgroup writeback (a cgroupv2 feature) and
memory.move_charge_at_immigrate (a cgroupv1 feature).
If the kernel is hacked to always claim inode switching and memcg
moving_account, then this script triggers lockup in less than a minute:
cd /mnt/cgroup/memory
mkdir a b
echo 1 > a/memory.move_charge_at_immigrate
echo 1 > b/memory.move_charge_at_immigrate
(
echo $BASHPID > a/cgroup.procs
while true; do
dd if=/dev/zero of=/mnt/big bs=1M count=256
done
) &
while true; do
sync
done &
sleep 1h &
SLEEP=$!
while true; do
echo $SLEEP > a/cgroup.procs
echo $SLEEP > b/cgroup.procs
done
The deadlock does not seem possible, so it's debatable if there's any
reason to modify the kernel. I suggest we should to prevent future
surprises. And Wang Long said "this deadlock occurs three times in our
environment", so there's more reason to apply this, even to stable.
Stable 4.4 has minor conflicts applying this patch. For a clean 4.4 patch
see "[PATCH for-4.4] writeback: safer lock nesting"
https://lkml.org/lkml/2018/4/11/146
Wang Long said "this deadlock occurs three times in our environment"
[gthelen@google.com: v4]
Link: http://lkml.kernel.org/r/20180411084653.254724-1-gthelen@google.com
[akpm@linux-foundation.org: comment tweaks, struct initialization simplification]
Change-Id: Ibb773e8045852978f6207074491d262f1b3fb613
Link: http://lkml.kernel.org/r/20180410005908.167976-1-gthelen@google.com
Fixes: 682aa8e1a6 ("writeback: implement unlocked_inode_to_wb transaction and use it for stat updates")
Signed-off-by: Greg Thelen <gthelen@google.com>
Reported-by: Wang Long <wanglong19@meituan.com>
Acked-by: Wang Long <wanglong19@meituan.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Tejun Heo <tj@kernel.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: <stable@vger.kernel.org> [v4.2+]
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
[natechancellor: Adjust context due to lack of b93b016313b3b]
Signed-off-by: Nathan Chancellor <natechancellor@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEZH8oZUiU471FcZm+ONu9yGCSaT4FAlo6KFYACgkQONu9yGCS
aT4mzw//cSnAjc7kuTtk96GKWat1bQExyb4scmsEkArfVKhoCy0Dhyr9yr4Y6+mX
6l2uUyQ70jhqOvinWIVuoDJoiZhtloudCe6ehmXm81xZsLacmelIC9NHGZ/vx/10
vC4BIZZgft5JiL4OSp/XTd0t++8maK5RUwp8cCqqTDeUyqHKNjUg+moMuJdjvRf7
4qGoBZ4lyijUU5V+WC98KSZSPncU6U1atA6k6Yvgu7oMFGembztERCx19Ka0JxA5
mzsmAH3TIhHUSGinDpTfW9x3Cmu0Dg3H7mQ0AaEVjhAi1oKTxxp0drLCZbeJUAXX
9QPJqr20XZWkuGX/yuy1vkcVo6kRfjaPYi1yyiFoEQ23hYXAaTJyiXC/CWv6kAkc
MOIXqHQgfegDAC33EzVunp/ue0sBwVAhFTpaTwbUiKJ+lpZY74mV+bjk6gZdlGJM
9TAOE66oAPNt6SM+5QC5mtO9cC03nCDIzbud5KXzdjYH8RBfIEvidxNv5qM6x8Hb
dJn6//nQzMTYIQFHja19Sqbt0xXq2lck5DrZZ+YnXlHr5JH1DzPQfqfmu8GD094e
H3oLDUmyBVnkI5jmgo3Xc+ZLArUMX7HhTyKSp+mXxRtGNulcbbQwaSWjEUoqYSzN
twMQPS+NKu+ZuubztP+7gOvyofmAAfcPX6yZpTnPyFKEnjyU3Uw=
=nCSn
-----END PGP SIGNATURE-----
Merge 4.9.71 into android-4.9
Changes in 4.9.71
mfd: fsl-imx25: Clean up irq settings during removal
crypto: rsa - fix buffer overread when stripping leading zeroes
crypto: hmac - require that the underlying hash algorithm is unkeyed
crypto: salsa20 - fix blkcipher_walk API usage
autofs: fix careless error in recent commit
tracing: Allocate mask_str buffer dynamically
USB: uas and storage: Add US_FL_BROKEN_FUA for another JMicron JMS567 ID
USB: core: prevent malicious bNumInterfaces overflow
usbip: fix stub_rx: get_pipe() to validate endpoint number
usb: add helper to extract bits 12:11 of wMaxPacketSize
usbip: fix stub_rx: harden CMD_SUBMIT path to handle malicious input
usbip: fix stub_send_ret_submit() vulnerability to null transfer_buffer
ceph: drop negative child dentries before try pruning inode's alias
usb: xhci: fix TDS for MTK xHCI1.1
Bluetooth: btusb: driver to enable the usb-wakeup feature
xhci: Don't add a virt_dev to the devs array before it's fully allocated
nfs: don't wait on commit in nfs_commit_inode() if there were no commit requests
sched/rt: Do not pull from current CPU if only one CPU to pull
eeprom: at24: change nvmem stride to 1
dmaengine: dmatest: move callback wait queue to thread context
ext4: fix fdatasync(2) after fallocate(2) operation
ext4: fix crash when a directory's i_size is too small
mac80211: Fix addition of mesh configuration element
usb: phy: isp1301: Add OF device ID table
KVM: nVMX: do not warn when MSR bitmap address is not backed
usb: xhci-mtk: check hcc_params after adding primary hcd
md-cluster: free md_cluster_info if node leave cluster
userfaultfd: shmem: __do_fault requires VM_FAULT_NOPAGE
userfaultfd: selftest: vm: allow to build in vm/ directory
net: initialize msg.msg_flags in recvfrom
bnxt_en: Ignore 0 value in autoneg supported speed from firmware.
net: bcmgenet: correct the RBUF_OVFL_CNT and RBUF_ERR_CNT MIB values
net: bcmgenet: correct MIB access of UniMAC RUNT counters
net: bcmgenet: reserved phy revisions must be checked first
net: bcmgenet: power down internal phy if open or resume fails
net: bcmgenet: synchronize irq0 status between the isr and task
net: bcmgenet: Power up the internal PHY before probing the MII
rxrpc: Wake up the transmitter if Rx window size increases on the peer
net/mlx5: Fix create autogroup prev initializer
net/mlx5: Don't save PCI state when PCI error is detected
iommu/io-pgtable-arm-v7s: Check for leaf entry before dereferencing it
drm/amdgpu: fix parser init error path to avoid crash in parser fini
NFSD: fix nfsd_minorversion(.., NFSD_AVAIL)
NFSD: fix nfsd_reset_versions for NFSv4.
Input: i8042 - add TUXEDO BU1406 (N24_25BU) to the nomux list
drm/omap: fix dmabuf mmap for dma_alloc'ed buffers
netfilter: bridge: honor frag_max_size when refragmenting
ASoC: rsnd: fix sound route path when using SRC6/SRC9
blk-mq: Fix tagset reinit in the presence of cpu hot-unplug
writeback: fix memory leak in wb_queue_work()
net: wimax/i2400m: fix NULL-deref at probe
dmaengine: Fix array index out of bounds warning in __get_unmap_pool()
irqchip/mvebu-odmi: Select GENERIC_MSI_IRQ_DOMAIN
net: Resend IGMP memberships upon peer notification.
mlxsw: reg: Fix SPVM max record count
mlxsw: reg: Fix SPVMLR max record count
qed: Align CIDs according to DORQ requirement
qed: Fix mapping leak on LL2 rx flow
qed: Fix interrupt flags on Rx LL2
drm: amd: remove broken include path
intel_th: pci: Add Gemini Lake support
openrisc: fix issue handling 8 byte get_user calls
ASoC: rcar: clear DE bit only in PDMACHCR when it stops
scsi: hpsa: update check for logical volume status
scsi: hpsa: limit outstanding rescans
scsi: hpsa: do not timeout reset operations
fjes: Fix wrong netdevice feature flags
drm/radeon/si: add dpm quirk for Oland
Drivers: hv: util: move waiting for release to hv_utils_transport itself
iwlwifi: mvm: cleanup pending frames in DQA mode
sched/deadline: Add missing update_rq_clock() in dl_task_timer()
sched/deadline: Make sure the replenishment timer fires in the next period
sched/deadline: Throttle a constrained deadline task activated after the deadline
sched/deadline: Use deadline instead of period when calculating overflow
mmc: mediatek: Fixed bug where clock frequency could be set wrong
drm/radeon: reinstate oland workaround for sclk
afs: Fix missing put_page()
afs: Populate group ID from vnode status
afs: Adjust mode bits processing
afs: Deal with an empty callback array
afs: Flush outstanding writes when an fd is closed
afs: Migrate vlocation fields to 64-bit
afs: Prevent callback expiry timer overflow
afs: Fix the maths in afs_fs_store_data()
afs: Invalid op ID should abort with RXGEN_OPCODE
afs: Better abort and net error handling
afs: Populate and use client modification time
afs: Fix page leak in afs_write_begin()
afs: Fix afs_kill_pages()
afs: Fix abort on signal while waiting for call completion
nvme-loop: fix a possible use-after-free when destroying the admin queue
nvmet: confirm sq percpu has scheduled and switched to atomic
nvmet-rdma: Fix a possible uninitialized variable dereference
net/mlx4_core: Avoid delays during VF driver device shutdown
net: mpls: Fix nexthop alive tracking on down events
rxrpc: Ignore BUSY packets on old calls
tty: don't panic on OOM in tty_set_ldisc()
tty: fix data race in tty_ldisc_ref_wait()
perf symbols: Fix symbols__fixup_end heuristic for corner cases
efi/esrt: Cleanup bad memory map log messages
NFSv4.1 respect server's max size in CREATE_SESSION
btrfs: add missing memset while reading compressed inline extents
target: Use system workqueue for ALUA transitions
target: fix ALUA transition timeout handling
target: fix race during implicit transition work flushes
Revert "x86/acpi: Set persistent cpuid <-> nodeid mapping when booting"
HID: cp2112: fix broken gpio_direction_input callback
sfc: don't warn on successful change of MAC
fbdev: controlfb: Add missing modes to fix out of bounds access
video: udlfb: Fix read EDID timeout
video: fbdev: au1200fb: Release some resources if a memory allocation fails
video: fbdev: au1200fb: Return an error code if a memory allocation fails
rtc: pcf8563: fix output clock rate
ASoC: Intel: Skylake: Fix uuid_module memory leak in failure case
dmaengine: ti-dma-crossbar: Correct am335x/am43xx mux value type
PCI/PME: Handle invalid data when reading Root Status
powerpc/powernv/cpufreq: Fix the frequency read by /proc/cpuinfo
PCI: Do not allocate more buses than available in parent
iommu/mediatek: Fix driver name
netfilter: ipvs: Fix inappropriate output of procfs
powerpc/opal: Fix EBUSY bug in acquiring tokens
powerpc/ipic: Fix status get and status clear
platform/x86: intel_punit_ipc: Fix resource ioremap warning
target/iscsi: Fix a race condition in iscsit_add_reject_from_cmd()
iscsi-target: fix memory leak in lio_target_tiqn_addtpg()
target:fix condition return in core_pr_dump_initiator_port()
target/file: Do not return error for UNMAP if length is zero
badblocks: fix wrong return value in badblocks_set if badblocks are disabled
iommu/amd: Limit the IOVA page range to the specified addresses
xfs: truncate pagecache before writeback in xfs_setattr_size()
arm-ccn: perf: Prevent module unload while PMU is in use
crypto: tcrypt - fix buffer lengths in test_aead_speed()
mm: Handle 0 flags in _calc_vm_trans() macro
clk: mediatek: add the option for determining PLL source clock
clk: imx6: refine hdmi_isfr's parent to make HDMI work on i.MX6 SoCs w/o VPU
clk: hi6220: mark clock cs_atb_syspll as critical
clk: tegra: Fix cclk_lp divisor register
ppp: Destroy the mutex when cleanup
ASoC: rsnd: rsnd_ssi_run_mods() needs to care ssi_parent_mod
thermal/drivers/step_wise: Fix temperature regulation misbehavior
scsi: scsi_debug: write_same: fix error report
GFS2: Take inode off order_write list when setting jdata flag
bcache: explicitly destroy mutex while exiting
bcache: fix wrong cache_misses statistics
Ib/hfi1: Return actual operational VLs in port info query
arm64: prevent regressions in compressed kernel image size when upgrading to binutils 2.27
btrfs: tests: Fix a memory leak in error handling path in 'run_test()'
platform/x86: hp_accel: Add quirk for HP ProBook 440 G4
nvme: use kref_get_unless_zero in nvme_find_get_ns
l2tp: cleanup l2tp_tunnel_delete calls
xfs: fix log block underflow during recovery cycle verification
xfs: fix incorrect extent state in xfs_bmap_add_extent_unwritten_real
RDMA/cxgb4: Declare stag as __be32
PCI: Detach driver before procfs & sysfs teardown on device remove
scsi: hpsa: cleanup sas_phy structures in sysfs when unloading
scsi: hpsa: destroy sas transport properties before scsi_host
powerpc/perf/hv-24x7: Fix incorrect comparison in memord
soc: mediatek: pwrap: fix compiler errors
tty fix oops when rmmod 8250
usb: musb: da8xx: fix babble condition handling
pinctrl: adi2: Fix Kconfig build problem
raid5: Set R5_Expanded on parity devices as well as data.
scsi: scsi_devinfo: Add REPORTLUN2 to EMC SYMMETRIX blacklist entry
IB/core: Fix calculation of maximum RoCE MTU
vt6655: Fix a possible sleep-in-atomic bug in vt6655_suspend
rtl8188eu: Fix a possible sleep-in-atomic bug in rtw_createbss_cmd
rtl8188eu: Fix a possible sleep-in-atomic bug in rtw_disassoc_cmd
scsi: sd: change manage_start_stop to bool in sysfs interface
scsi: sd: change allow_restart to bool in sysfs interface
scsi: bfa: integer overflow in debugfs
udf: Avoid overflow when session starts at large offset
macvlan: Only deliver one copy of the frame to the macvlan interface
RDMA/cma: Avoid triggering undefined behavior
IB/ipoib: Grab rtnl lock on heavy flush when calling ndo_open/stop
icmp: don't fail on fragment reassembly time exceeded
ath9k: fix tx99 potential info leak
Linux 4.9.71
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
[ Upstream commit 4a3a485b1ed0e109718cc8c9d094fa0f552de9b2 ]
When WB_registered flag is not set, wb_queue_work() skips queuing the
work, but does not perform the necessary clean up. In particular, if
work->auto_free is true, it should free the memory.
The leak condition can be reprouced by following these steps:
mount /dev/sdb /mnt/sdb
/* In qemu console: device_del sdb */
umount /dev/sdb
Above will result in a wb_queue_work() call on an unregistered wb and
thus leak memory.
Reported-by: John Sperbeck <jsperbeck@google.com>
Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Jens Axboe <axboe@fb.com>
Signed-off-by: Sasha Levin <alexander.levin@verizon.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
I've found funny live-lock between raid10 barriers during resync and
memory controller hard limits. Inside mpage_readpages() task holds on to
its plug bio which blocks the barrier in raid10. Its memory cgroup have
no free memory thus the task goes into reclaimer but all reclaimable
pages are dirty and cannot be written because raid10 is rebuilding and
stuck on the barrier.
Common flush of such IO in schedule() never happens, because the caller
doesn't go to sleep.
Lock is 'live' because changing memory limit or killing tasks which
holds that stuck bio unblock whole progress.
That was what happened in 3.18.x but I see no difference in upstream
logic. Theoretically this might happen even without memory cgroup.
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Signed-off-by: Jens Axboe <axboe@fb.com>
There are now a number of accounting oddities such as mapped file pages
being accounted for on the node while the total number of file pages are
accounted on the zone. This can be coped with to some extent but it's
confusing so this patch moves the relevant file-based accounted. Due to
throttling logic in the page allocator for reliable OOM detection, it is
still necessary to track dirty and writeback pages on a per-zone basis.
[mgorman@techsingularity.net: fix NR_ZONE_WRITE_PENDING accounting]
Link: http://lkml.kernel.org/r/1468404004-5085-5-git-send-email-mgorman@techsingularity.net
Link: http://lkml.kernel.org/r/1467970510-21195-20-git-send-email-mgorman@techsingularity.net
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Acked-by: Michal Hocko <mhocko@suse.com>
Cc: Hillf Danton <hillf.zj@alibaba-inc.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Rik van Riel <riel@surriel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
The per-sb inode writeback list tracks inodes currently under writeback
to facilitate efficient sync processing. In particular, it ensures that
sync only needs to walk through a list of inodes that were cleaned by
the sync.
Add a couple tracepoints to help identify when inodes are added/removed
to and from the writeback lists. Piggyback off of the writeback
lazytime tracepoint template as it already tracks the relevant inode
information.
Link: http://lkml.kernel.org/r/1466594593-6757-3-git-send-email-bfoster@redhat.com
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: Dave Chinner <dchinner@redhat.com>
cc: Josef Bacik <jbacik@fb.com>
Cc: Holger Hoffstätte <holger.hoffstaette@applied-asynchrony.com>
Cc: Al Viro <viro@ZenIV.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
wait_sb_inodes() currently does a walk of all inodes in the filesystem
to find dirty one to wait on during sync. This is highly inefficient
and wastes a lot of CPU when there are lots of clean cached inodes that
we don't need to wait on.
To avoid this "all inode" walk, we need to track inodes that are
currently under writeback that we need to wait for. We do this by
adding inodes to a writeback list on the sb when the mapping is first
tagged as having pages under writeback. wait_sb_inodes() can then walk
this list of "inodes under IO" and wait specifically just for the inodes
that the current sync(2) needs to wait for.
Define a couple helpers to add/remove an inode from the writeback list
and call them when the overall mapping is tagged for or cleared from
writeback. Update wait_sb_inodes() to walk only the inodes under
writeback due to the sync.
With this change, filesystem sync times are significantly reduced for
fs' with largely populated inode caches and otherwise no other work to
do. For example, on a 16xcpu 2GHz x86-64 server, 10TB XFS filesystem
with a ~10m entry inode cache, sync times are reduced from ~7.3s to less
than 0.1s when the filesystem is fully clean.
Link: http://lkml.kernel.org/r/1466594593-6757-2-git-send-email-bfoster@redhat.com
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Tested-by: Holger Hoffstätte <holger.hoffstaette@applied-asynchrony.com>
Cc: Al Viro <viro@ZenIV.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
PAGE_CACHE_{SIZE,SHIFT,MASK,ALIGN} macros were introduced *long* time
ago with promise that one day it will be possible to implement page
cache with bigger chunks than PAGE_SIZE.
This promise never materialized. And unlikely will.
We have many places where PAGE_CACHE_SIZE assumed to be equal to
PAGE_SIZE. And it's constant source of confusion on whether
PAGE_CACHE_* or PAGE_* constant should be used in a particular case,
especially on the border between fs and mm.
Global switching to PAGE_CACHE_SIZE != PAGE_SIZE would cause to much
breakage to be doable.
Let's stop pretending that pages in page cache are special. They are
not.
The changes are pretty straight-forward:
- <foo> << (PAGE_CACHE_SHIFT - PAGE_SHIFT) -> <foo>;
- <foo> >> (PAGE_CACHE_SHIFT - PAGE_SHIFT) -> <foo>;
- PAGE_CACHE_{SIZE,SHIFT,MASK,ALIGN} -> PAGE_{SIZE,SHIFT,MASK,ALIGN};
- page_cache_get() -> get_page();
- page_cache_release() -> put_page();
This patch contains automated changes generated with coccinelle using
script below. For some reason, coccinelle doesn't patch header files.
I've called spatch for them manually.
The only adjustment after coccinelle is revert of changes to
PAGE_CAHCE_ALIGN definition: we are going to drop it later.
There are few places in the code where coccinelle didn't reach. I'll
fix them manually in a separate patch. Comments and documentation also
will be addressed with the separate patch.
virtual patch
@@
expression E;
@@
- E << (PAGE_CACHE_SHIFT - PAGE_SHIFT)
+ E
@@
expression E;
@@
- E >> (PAGE_CACHE_SHIFT - PAGE_SHIFT)
+ E
@@
@@
- PAGE_CACHE_SHIFT
+ PAGE_SHIFT
@@
@@
- PAGE_CACHE_SIZE
+ PAGE_SIZE
@@
@@
- PAGE_CACHE_MASK
+ PAGE_MASK
@@
expression E;
@@
- PAGE_CACHE_ALIGN(E)
+ PAGE_ALIGN(E)
@@
expression E;
@@
- page_cache_get(E)
+ get_page(E)
@@
expression E;
@@
- page_cache_release(E)
+ put_page(E)
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
When cgroup writeback is in use, there can be multiple wb's
(bdi_writeback's) per bdi and an inode may switch among them
dynamically. In a couple places, the wrong wb was used leading to
performing operations on the wrong list under the wrong lock
corrupting the io lists.
* writeback_single_inode() was taking @wb parameter and used it to
remove the inode from io lists if it becomes clean after writeback.
The callers of this function were always passing in the root wb
regardless of the actual wb that the inode was associated with,
which could also change while writeback is in progress.
Fix it by dropping the @wb parameter and using
inode_to_wb_and_lock_list() to determine and lock the associated wb.
* After writeback_sb_inodes() writes out an inode, it re-locks @wb and
inode to remove it from or move it to the right io list. It assumes
that the inode is still associated with @wb; however, the inode may
have switched to another wb while writeback was in progress.
Fix it by using inode_to_wb_and_lock_list() to determine and lock
the associated wb after writeback is complete. As the function
requires the original @wb->list_lock locked for the next iteration,
in the unlikely case where the inode has changed association, switch
the locks.
Kudos to Tahsin for pinpointing these subtle breakages.
Signed-off-by: Tejun Heo <tj@kernel.org>
Fixes: d10c809552 ("writeback: implement foreign cgroup inode bdi_writeback switching")
Link: http://lkml.kernel.org/g/CAAeU0aMYeM_39Y2+PaRvyB1nqAPYZSNngJ1eBRmrxn7gKAt2Mg@mail.gmail.com
Reported-and-diagnosed-by: Tahsin Erdogan <tahsin@google.com>
Tested-by: Tahsin Erdogan <tahsin@google.com>
Cc: stable@vger.kernel.org # v4.2+
Signed-off-by: Jens Axboe <axboe@fb.com>
locked_inode_to_wb_and_lock_list() wb_get()'s the wb associated with
the target inode, unlocks inode, locks the wb's list_lock and verifies
that the inode is still associated with the wb. To prevent the wb
going away between dropping inode lock and acquiring list_lock, the wb
is pinned while inode lock is held. The wb reference is put right
after acquiring list_lock citing that the wb won't be dereferenced
anymore.
This isn't true. If the inode is still associated with the wb, the
inode has reference and it's safe to return the wb; however, if inode
has been switched, the wb still needs to be unlocked which is a
dereference and can lead to use-after-free if it it races with wb
destruction.
Fix it by putting the reference after releasing list_lock.
Signed-off-by: Tejun Heo <tj@kernel.org>
Fixes: 87e1d789bf ("writeback: implement [locked_]inode_to_wb_and_lock_list()")
Cc: stable@vger.kernel.org # v4.2+
Tested-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
If cgroup writeback is in use, inodes can be scheduled for
asynchronous wb switching. Before 5ff8eaac16 ("writeback: keep
superblock pinned during cgroup writeback association switches"), this
could race with umount leading to super_block being destroyed while
inodes are pinned for wb switching. 5ff8eaac16 fixed it by bumping
s_active while wb switches are in flight; however, this allowed
in-flight wb switches to make umounts asynchronous when the userland
expected synchronosity - e.g. fsck immediately following umount may
fail because the device is still busy.
This patch removes the problematic super_block pinning and instead
makes generic_shutdown_super() flush in-flight wb switches. wb
switches are now executed on a dedicated isw_wq so that they can be
flushed and isw_nr_in_flight keeps track of the number of in-flight wb
switches so that flushing can be avoided in most cases.
v2: Move cgroup_writeback_umount() further below and add MS_ACTIVE
check in inode_switch_wbs() as Jan an Al suggested.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-by: Tahsin Erdogan <tahsin@google.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Al Viro <viro@ZenIV.linux.org.uk>
Link: http://lkml.kernel.org/g/CAAeU0aNCq7LGODvVGRU-oU_o-6enii5ey0p1c26D1ZzYwkDc5A@mail.gmail.com
Fixes: 5ff8eaac16 ("writeback: keep superblock pinned during cgroup writeback association switches")
Cc: stable@vger.kernel.org #v4.5
Reviewed-by: Jan Kara <jack@suse.cz>
Tested-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
If cgroup writeback is in use, an inode is associated with a cgroup
for writeback. If the inode's main dirtier changes to another cgroup,
the association gets updated asynchronously. Nothing was pinning the
superblock while such switches are in progress and superblock could go
away while async switching is pending or in progress leading to
crashes like the following.
kernel BUG at fs/jbd2/transaction.c:319!
invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
CPU: 1 PID: 29158 Comm: kworker/1:10 Not tainted 4.5.0-rc3 #51
Hardware name: Google Google, BIOS Google 01/01/2011
Workqueue: events inode_switch_wbs_work_fn
task: ffff880213dbbd40 ti: ffff880209264000 task.ti: ffff880209264000
RIP: 0010:[<ffffffff803e6922>] [<ffffffff803e6922>] start_this_handle+0x382/0x3e0
RSP: 0018:ffff880209267c30 EFLAGS: 00010202
...
Call Trace:
[<ffffffff803e6be4>] jbd2__journal_start+0xf4/0x190
[<ffffffff803cfc7e>] __ext4_journal_start_sb+0x4e/0x70
[<ffffffff803b31ec>] ext4_evict_inode+0x12c/0x3d0
[<ffffffff8035338b>] evict+0xbb/0x190
[<ffffffff80354190>] iput+0x130/0x190
[<ffffffff80360223>] inode_switch_wbs_work_fn+0x343/0x4c0
[<ffffffff80279819>] process_one_work+0x129/0x300
[<ffffffff80279b16>] worker_thread+0x126/0x480
[<ffffffff8027ed14>] kthread+0xc4/0xe0
[<ffffffff809771df>] ret_from_fork+0x3f/0x70
Fix it by bumping s_active while cgroup association switching is in
flight.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-and-tested-by: Tahsin Erdogan <tahsin@google.com>
Link: http://lkml.kernel.org/g/CAAeU0aNCq7LGODvVGRU-oU_o-6enii5ey0p1c26D1ZzYwkDc5A@mail.gmail.com
Fixes: d10c809552 ("writeback: implement foreign cgroup inode bdi_writeback switching")
Cc: stable@vger.kernel.org #v4.5+
Signed-off-by: Jens Axboe <axboe@fb.com>
In earlier versions, mem_cgroup_css_from_page() could return non-root
css on a legacy hierarchy which can go away and required rcu locking;
however, the eventual version simply returns the root cgroup if memcg is
on a legacy hierarchy and thus doesn't need rcu locking around or in it.
Remove spurious rcu lockings.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-by: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Vladimir Davydov <vdavydov@virtuozzo.com>
Cc: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Fix kernel-doc warnings in fs/fs-writeback.c by moving a #define macro to
after the function's opening brace. Also #undef this macro at the end of
the function.
../fs/fs-writeback.c:1984: warning: Excess function parameter 'inode' description in 'I_DIRTY_INODE'
../fs/fs-writeback.c:1984: warning: Excess function parameter 'flags' description in 'I_DIRTY_INODE'
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
filemap_fdatawait() is a function to wait for on-going writeback to
complete but also consume and clear error status of the mapping set during
writeback.
The latter functionality is critical for applications to detect writeback
error with system calls like fsync(2)/fdatasync(2).
However filemap_fdatawait() is also used by sync(2) or FIFREEZE ioctl,
which don't check error status of individual mappings.
As a result, fsync() may not be able to detect writeback error if events
happen in the following order:
Application System admin
----------------------------------------------------------
write data on page cache
Run sync command
writeback completes with error
filemap_fdatawait() clears error
fsync returns success
(but the data is not on disk)
This patch adds filemap_fdatawait_keep_errors() for call sites where
writeback error is not handled so that they don't clear error status.
Signed-off-by: Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
Acked-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Tejun Heo <tj@kernel.org>
Cc: Fengguang Wu <fengguang.wu@gmail.com>
Cc: Dave Chinner <david@fromorbit.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
bdi_split_work_to_wbs() uses list_for_each_entry_rcu_continue()
to walk @bdi->wb_list. To set up the initial iteration
condition, it uses list_entry_rcu() to calculate the entry
pointer corresponding to the list head; however, this isn't an
actual RCU dereference and using list_entry_rcu() for it ended
up breaking a proposed list_entry_rcu() change because it was
feeding an non-lvalue pointer into the macro.
Don't use the RCU variant for simple pointer offsetting. Use
list_entry() instead.
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Darren Hart <dvhart@linux.intel.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Dipankar Sarma <dipankar@in.ibm.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Patrick Marlier <patrick.marlier@gmail.com>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: pranith kumar <bobby.prani@gmail.com>
Link: http://lkml.kernel.org/r/20151027051939.GA19355@mtj.duckdns.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
bdi_for_each_wb() is used in several places to wake up or issue
writeback work items to all wb's (bdi_writeback's) on a given bdi.
The iteration is performed by walking bdi->cgwb_tree; however, the
tree only indexes wb's which are currently active.
For example, when a memcg gets associated with a different blkcg, the
old wb is removed from the tree so that the new one can be indexed.
The old wb starts dying from then on but will linger till all its
inodes are drained. As these dying wb's may still host dirty inodes,
writeback operations which affect all wb's must include them.
bdi_for_each_wb() skipping dying wb's led to sync(2) missing and
failing to sync the inodes belonging to those wb's.
This patch adds a RCU protected @bdi->wb_list which lists all wb's
beloinging to that bdi. wb's are added on creation and removed on
release rather than on the start of destruction. bdi_for_each_wb()
usages are replaced with list_for_each[_continue]_rcu() iterations
over @bdi->wb_list and bdi_for_each_wb() and its helpers are removed.
v2: Updated as per Jan. last_wb ref leak in bdi_split_work_to_wbs()
fixed and unnecessary list head severing in cgwb_bdi_destroy()
removed.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-and-tested-by: Artem Bityutskiy <dedekind1@gmail.com>
Fixes: ebe41ab0c7 ("writeback: implement bdi_for_each_wb()")
Link: http://lkml.kernel.org/g/1443012552.19983.209.camel@gmail.com
Cc: Jan Kara <jack@suse.cz>
Signed-off-by: Jens Axboe <axboe@fb.com>
wakeup_dirtytime_writeback() walks and wakes up all wb's of all bdi's;
unfortunately, it was always waking up bdi->wb instead of the wb being
walked. Fix it.
Signed-off-by: Tejun Heo <tj@kernel.org>
Fixes: 001fe6f617 ("writeback: make wakeup_dirtytime_writeback() handle multiple bdi_writeback's")
Reviewed-by: Jan Kara <jack@suse.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Commit 505a666ee3 ("writeback: plug writeback in wb_writeback() and
writeback_inodes_wb()") has us holding a plug during writeback_sb_inodes,
which increases the merge rate when relatively contiguous small files
are written by the filesystem. It helps both on flash and spindles.
For an fs_mark workload creating 4K files in parallel across 8 drives,
this commit improves performance ~9% more by unplugging before calling
cond_resched(). cond_resched() doesn't trigger an implicit unplug, so
explicitly getting the IO down to the device before scheduling reduces
latencies for anyone waiting on clean pages.
It also cuts down on how often we use kblockd to unplug, which means
less work bouncing from one workqueue to another.
Many more details about how we got here:
https://lkml.org/lkml/2015/9/11/570
Signed-off-by: Chris Mason <clm@fb.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
We had to revert the pluggin in writeback_sb_inodes() because the
wb->list_lock is held, but we could easily plug at a higher level before
taking that lock, and unplug after releasing it. This does that.
Chris will run performance numbers, just to verify that this approach is
comparable to the alternative (we could just drop and re-take the lock
around the blk_finish_plug() rather than these two commits.
I'd have preferred waiting for actual performance numbers before picking
one approach over the other, but I don't want to release rc1 with the
known "sleeping function called from invalid context" issue, so I'll
pick this cleanup version for now. But if the numbers show that we
really want to plug just at the writeback_sb_inodes() level, and we
should just play ugly games with the spinlock, we'll switch to that.
Cc: Chris Mason <clm@fb.com>
Cc: Josef Bacik <jbacik@fb.com>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Neil Brown <neilb@suse.de>
Cc: Jan Kara <jack@suse.cz>
Cc: Christoph Hellwig <hch@lst.de>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This reverts commit d353d7587d.
Doing the block layer plug/unplug inside writeback_sb_inodes() is
broken, because that function is actually called with a spinlock held:
wb->list_lock, as pointed out by Chris Mason.
Chris suggested just dropping and re-taking the spinlock around the
blk_finish_plug() call (the plgging itself can happen under the
spinlock), and that would technically work, but is just disgusting.
We do something fairly similar - but not quite as disgusting because we
at least have a better reason for it - in writeback_single_inode(), so
it's not like the caller can depend on the lock being held over the
call, but in this case there just isn't any good reason for that
"release and re-take the lock" pattern.
[ In general, we should really strive to avoid the "release and retake"
pattern for locks, because in the general case it can easily cause
subtle bugs when the caller caches any state around the call that
might be invalidated by dropping the lock even just temporarily. ]
But in this case, the plugging should be easy to just move up to the
callers before the spinlock is taken, which should even improve the
effectiveness of the plug. So there is really no good reason to play
games with locking here.
I'll send off a test-patch so that Dave Chinner can verify that that
plug movement works. In the meantime this just reverts the problematic
commit and adds a comment to the function so that we hopefully don't
make this mistake again.
Reported-by: Chris Mason <clm@fb.com>
Cc: Josef Bacik <jbacik@fb.com>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Neil Brown <neilb@suse.de>
Cc: Jan Kara <jack@suse.cz>
Cc: Christoph Hellwig <hch@lst.de>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Pull blk-cg updates from Jens Axboe:
"A bit later in the cycle, but this has been in the block tree for a a
while. This is basically four patchsets from Tejun, that improve our
buffered cgroup writeback. It was dependent on the other cgroup
changes, but they went in earlier in this cycle.
Series 1 is set of 5 patches that has cgroup writeback updates:
- bdi_writeback iteration fix which could lead to some wb's being
skipped or repeated during e.g. sync under memory pressure.
- Simplification of wb work wait mechanism.
- Writeback tracepoints updated to report cgroup.
Series 2 is is a set of updates for the CFQ cgroup writeback handling:
cfq has always charged all async IOs to the root cgroup. It didn't
have much choice as writeback didn't know about cgroups and there
was no way to tell who to blame for a given writeback IO.
writeback finally grew support for cgroups and now tags each
writeback IO with the appropriate cgroup to charge it against.
This patchset updates cfq so that it follows the blkcg each bio is
tagged with. Async cfq_queues are now shared across cfq_group,
which is per-cgroup, instead of per-request_queue cfq_data. This
makes all IOs follow the weight based IO resource distribution
implemented by cfq.
- Switched from GFP_ATOMIC to GFP_NOWAIT as suggested by Jeff.
- Other misc review points addressed, acks added and rebased.
Series 3 is the blkcg policy cleanup patches:
This patchset contains assorted cleanups for blkcg_policy methods
and blk[c]g_policy_data handling.
- alloc/free added for blkg_policy_data. exit dropped.
- alloc/free added for blkcg_policy_data.
- blk-throttle's async percpu allocation is replaced with direct
allocation.
- all methods now take blk[c]g_policy_data instead of blkcg_gq or
blkcg.
And finally, series 4 is a set of patches cleaning up the blkcg stats
handling:
blkcg's stats have always been somwhat of a mess. This patchset
tries to improve the situation a bit.
- The following patches added to consolidate blkcg entry point and
blkg creation. This is in itself is an improvement and helps
colllecting common stats on bio issue.
- per-blkg stats now accounted on bio issue rather than request
completion so that bio based and request based drivers can behave
the same way. The issue was spotted by Vivek.
- cfq-iosched implements custom recursive stats and blk-throttle
implements custom per-cpu stats. This patchset make blkcg core
support both by default.
- cfq-iosched and blk-throttle keep track of the same stats
multiple times. Unify them"
* 'for-4.3/blkcg' of git://git.kernel.dk/linux-block: (45 commits)
blkcg: use CGROUP_WEIGHT_* scale for io.weight on the unified hierarchy
blkcg: s/CFQ_WEIGHT_*/CFQ_WEIGHT_LEGACY_*/
blkcg: implement interface for the unified hierarchy
blkcg: misc preparations for unified hierarchy interface
blkcg: separate out tg_conf_updated() from tg_set_conf()
blkcg: move body parsing from blkg_conf_prep() to its callers
blkcg: mark existing cftypes as legacy
blkcg: rename subsystem name from blkio to io
blkcg: refine error codes returned during blkcg configuration
blkcg: remove unnecessary NULL checks from __cfqg_set_weight_device()
blkcg: reduce stack usage of blkg_rwstat_recursive_sum()
blkcg: remove cfqg_stats->sectors
blkcg: move io_service_bytes and io_serviced stats into blkcg_gq
blkcg: make blkg_[rw]stat_recursive_sum() to be able to index into blkcg_gq
blkcg: make blkcg_[rw]stat per-cpu
blkcg: add blkg_[rw]stat->aux_cnt and replace cfq_group->dead_stats with it
blkcg: consolidate blkg creation in blkcg_bio_issue_check()
blk-throttle: improve queue bypass handling
blkcg: move root blkg lookup optimization from throtl_lookup_tg() to __blkg_lookup()
blkcg: inline [__]blkg_lookup()
...
Pull vfs updates from Al Viro:
"In this one:
- d_move fixes (Eric Biederman)
- UFS fixes (me; locking is mostly sane now, a bunch of bugs in error
handling ought to be fixed)
- switch of sb_writers to percpu rwsem (Oleg Nesterov)
- superblock scalability (Josef Bacik and Dave Chinner)
- swapon(2) race fix (Hugh Dickins)"
* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs: (65 commits)
vfs: Test for and handle paths that are unreachable from their mnt_root
dcache: Reduce the scope of i_lock in d_splice_alias
dcache: Handle escaped paths in prepend_path
mm: fix potential data race in SyS_swapon
inode: don't softlockup when evicting inodes
inode: rename i_wb_list to i_io_list
sync: serialise per-superblock sync operations
inode: convert inode_sb_list_lock to per-sb
inode: add hlist_fake to avoid the inode hash lock in evict
writeback: plug writeback at a high level
change sb_writers to use percpu_rw_semaphore
shift percpu_counter_destroy() into destroy_super_work()
percpu-rwsem: kill CONFIG_PERCPU_RWSEM
percpu-rwsem: introduce percpu_rwsem_release() and percpu_rwsem_acquire()
percpu-rwsem: introduce percpu_down_read_trylock()
document rwsem_release() in sb_wait_write()
fix the broken lockdep logic in __sb_start_write()
introduce __sb_writers_{acquired,release}() helpers
ufs_inode_get{frag,block}(): get rid of 'phys' argument
ufs_getfrag_block(): tidy up a bit
...
e79729123f ("writeback: don't issue wb_writeback_work if clean")
updated writeback path to avoid kicking writeback work items if there
are no inodes to be written out; unfortunately, the avoidance logic
was too aggressive and broke sync_inodes_sb().
* sync_inodes_sb() must write out I_DIRTY_TIME inodes but I_DIRTY_TIME
inodes dont't contribute to bdi/wb_has_dirty_io() tests and were
being skipped over.
* inodes are taken off wb->b_dirty/io/more_io lists after writeback
starts on them. sync_inodes_sb() skipping wait_sb_inodes() when
bdi_has_dirty_io() breaks it by making it return while writebacks
are in-flight.
This patch fixes the breakages by
* Removing bdi_has_dirty_io() shortcut from bdi_split_work_to_wbs().
The callers are already testing the condition.
* Removing bdi_has_dirty_io() shortcut from sync_inodes_sb() so that
it always calls into bdi_split_work_to_wbs() and wait_sb_inodes().
* Making bdi_split_work_to_wbs() consider the b_dirty_time list for
WB_SYNC_ALL writebacks.
Kudos to Eryu, Dave and Jan for tracking down the issue.
Signed-off-by: Tejun Heo <tj@kernel.org>
Fixes: e79729123f ("writeback: don't issue wb_writeback_work if clean")
Link: http://lkml.kernel.org/g/20150812101204.GE17933@dhcp-13-216.nay.redhat.com
Reported-and-bisected-by: Eryu Guan <eguan@redhat.com>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Jan Kara <jack@suse.com>
Cc: Ted Ts'o <tytso@google.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
The following tracepoints are updated to report the cgroup used during
cgroup writeback.
* writeback_write_inode[_start]
* writeback_queue
* writeback_exec
* writeback_start
* writeback_written
* writeback_wait
* writeback_nowork
* writeback_wake_background
* wbc_writepage
* writeback_queue_io
* bdi_dirty_ratelimit
* balance_dirty_pages
* writeback_sb_inodes_requeue
* writeback_single_inode[_start]
Note that writeback_bdi_register is separated out from writeback_class
as reporting cgroup doesn't make sense to it. Tracepoints which take
bdi are updated to take bdi_writeback instead.
Signed-off-by: Tejun Heo <tj@kernel.org>
Suggested-by: Jan Kara <jack@suse.cz>
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Jens Axboe <axboe@fb.com>
wb_writeback_work->single_wait/done are used for the wait mechanism
for synchronous wb_work (wb_writeback_work) items which are issued
when bdi_split_work_to_wbs() fails to allocate memory for asynchronous
wb_work items; however, there's no reason to use a separate wait
mechanism for this. bdi_split_work_to_wbs() can simply use on-stack
fallback wb_work item and separate wb_completion to wait for it.
This patch removes wb_work->single_wait/done and the related code and
make bdi_split_work_to_wbs() use on-stack fallback wb_work and
wb_completion instead.
Signed-off-by: Tejun Heo <tj@kernel.org>
Suggested-by: Jan Kara <jack@suse.cz>
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Jens Axboe <axboe@fb.com>
wb's (bdi_writeback's) are currently keyed by memcg ID; however, in an
earlier implementation, wb's were keyed by blkcg ID.
bdi_for_each_wb() walks bdi->cgwb_tree in the ascending ID order and
allows iterations to start from an arbitrary ID which is used to
interrupt and resume iterations.
Unfortunately, while changing wb to be keyed by memcg ID instead of
blkcg, bdi_for_each_wb() was missed and is still assuming that wb's
are keyed by blkcg ID. This doesn't affect iterations which don't get
interrupted but bdi_split_work_to_wbs() makes use of iteration
resuming on allocation failures and thus may incorrectly skip or
repeat wb's.
Fix it by changing bdi_for_each_wb() to take memcg IDs instead of
blkcg IDs and updating bdi_split_work_to_wbs() accordingly.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jens Axboe <axboe@fb.com>
There's a small consistency problem between the inode and writeback
naming. Writeback calls the "for IO" inode queues b_io and
b_more_io, but the inode calls these the "writeback list" or
i_wb_list. This makes it hard to an new "under writeback" list to
the inode, or call it an "under IO" list on the bdi because either
way we'll have writeback on IO and IO on writeback and it'll just be
confusing. I'm getting confused just writing this!
So, rename the inode "for IO" list variable to i_io_list so we can
add a new "writeback list" in a subsequent patch.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Josef Bacik <jbacik@fb.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Tested-by: Dave Chinner <dchinner@redhat.com>
When competing sync(2) calls walk the same filesystem, they need to
walk the list of inodes on the superblock to find all the inodes
that we need to wait for IO completion on. However, when multiple
wait_sb_inodes() calls do this at the same time, they contend on the
the inode_sb_list_lock and the contention causes system wide
slowdowns. In effect, concurrent sync(2) calls can take longer and
burn more CPU than if they were serialised.
Stop the worst of the contention by adding a per-sb mutex to wrap
around wait_sb_inodes() so that we only execute one sync(2) IO
completion walk per superblock superblock at a time and hence avoid
contention being triggered by concurrent sync(2) calls.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Josef Bacik <jbacik@fb.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Tested-by: Dave Chinner <dchinner@redhat.com>
The process of reducing contention on per-superblock inode lists
starts with moving the locking to match the per-superblock inode
list. This takes the global lock out of the picture and reduces the
contention problems to within a single filesystem. This doesn't get
rid of contention as the locks still have global CPU scope, but it
does isolate operations on different superblocks form each other.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Josef Bacik <jbacik@fb.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Tested-by: Dave Chinner <dchinner@redhat.com>
Doing writeback on lots of little files causes terrible IOPS storms
because of the per-mapping writeback plugging we do. This
essentially causes imeediate dispatch of IO for each mapping,
regardless of the context in which writeback is occurring.
IOWs, running a concurrent write-lots-of-small 4k files using fsmark
on XFS results in a huge number of IOPS being issued for data
writes. Metadata writes are sorted and plugged at a high level by
XFS, so aggregate nicely into large IOs. However, data writeback IOs
are dispatched in individual 4k IOs, even when the blocks of two
consecutively written files are adjacent.
Test VM: 8p, 8GB RAM, 4xSSD in RAID0, 100TB sparse XFS filesystem,
metadata CRCs enabled.
Kernel: 3.10-rc5 + xfsdev + my 3.11 xfs queue (~70 patches)
Test:
$ ./fs_mark -D 10000 -S0 -n 10000 -s 4096 -L 120 -d
/mnt/scratch/0 -d /mnt/scratch/1 -d /mnt/scratch/2 -d
/mnt/scratch/3 -d /mnt/scratch/4 -d /mnt/scratch/5 -d
/mnt/scratch/6 -d /mnt/scratch/7
Result:
wall sys create rate Physical write IO
time CPU (avg files/s) IOPS Bandwidth
----- ----- ------------ ------ ---------
unpatched 6m56s 15m47s 24,000+/-500 26,000 130MB/s
patched 5m06s 13m28s 32,800+/-600 1,500 180MB/s
improvement -26.44% -14.68% +36.67% -94.23% +38.46%
If I use zero length files, this workload at about 500 IOPS, so
plugging drops the data IOs from roughly 25,500/s to 1000/s.
3 lines of code, 35% better throughput for 15% less CPU.
The benefits of plugging at this layer are likely to be higher for
spinning media as the IO patterns for this workload are going make a
much bigger difference on high IO latency devices.....
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Josef Bacik <jbacik@fb.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Tested-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
bio_associate_blkcg(), bio_associate_current() and wbc_account_io()
are used to implement cgroup writeback support for filesystems and
thus need to be exported. Export them.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Jens Axboe <axboe@fb.com>
Currently, even when a filesystem doesn't set the FS_CGROUP_WRITEBACK
flag, if the filesystem uses wbc_init_bio() and wbc_account_io(), the
foreign inode detection and migration logic still ends up activating
cgroup writeback which is unexpected. This patch ensures that the
foreign inode detection logic stays disabled when inode_cgwb_enabled()
is false by not associating writeback_control's with bdi_writeback's.
This also avoids unnecessary operations in wbc_init_bio(),
wbc_account_io() and wbc_detach_inode() for filesystems which don't
support cgroup writeback.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jens Axboe <axboe@fb.com>
For the purpose of foreign inode detection, wb's (bdi_writeback's) are
identified by the associated memcg ID. As we create a separate wb for
each memcg, this is enough to identify the active wb's; however, when
blkcg is enabled or disabled higher up in the hierarchy, the mapping
between memcg and blkcg changes which in turn creates a new wb to
service the new mapping. The old wb is unlinked from index and
released after all references are drained. The foreign inode
detection logic can't detect this condition because both the old and
new wb's point to the same memcg and thus never decides to move inodes
attached to the old wb to the new one.
This patch adds logic to initiate switching immediately in
wbc_attach_and_unlock_inode() if the associated wb is dying. We can
make the usual foreign detection logic to distinguish the different
wb's mapped to the memcg but the dying wb is never gonna be in active
service again and there's no point in tracking the usage history and
reaching the switch verdict after enough data points are collected.
It's already known that the wb has to be switched.
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Jan Kara <jack@suse.cz>
Cc: Wu Fengguang <fengguang.wu@intel.com>
Cc: Greg Thelen <gthelen@google.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
As concurrent write sharing of an inode is expected to be very rare
and memcg only tracks page ownership on first-use basis severely
confining the usefulness of such sharing, cgroup writeback tracks
ownership per-inode. While the support for concurrent write sharing
of an inode is deemed unnecessary, an inode being written to by
different cgroups at different points in time is a lot more common,
and, more importantly, charging only by first-use can too readily lead
to grossly incorrect behaviors (single foreign page can lead to
gigabytes of writeback to be incorrectly attributed).
To resolve this issue, cgroup writeback detects the majority dirtier
of an inode and transfers the ownership to it. The previous patches
implemented the foreign condition detection mechanism and laid the
groundwork. This patch implements the actual switching.
With the previously implemented [unlocked_]inode_to_wb_and_list_lock()
and wb stat transaction, grabbing wb->list_lock, inode->i_lock and
mapping->tree_lock gives us full exclusion against all wb operations
on the target inode. inode_switch_wb_work_fn() grabs all the locks
and transfers the inode atomically along with its RECLAIMABLE and
WRITEBACK stats.
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Jan Kara <jack@suse.cz>
Cc: Wu Fengguang <fengguang.wu@intel.com>
Cc: Greg Thelen <gthelen@google.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Greg Kroah-Hartman
Tejun Heo
Jiufei Xue
Jan Kara
Greg Thelen
Tahsin Erdogan
San Mehat
Konstantin Khlebnikov
Mel Gorman
Brian Foster
Dave Chinner
Tetsuo Handa
Kirill A. Shutemov
Randy Dunlap
Junichi Nomura
Chris Mason
Linus Torvalds