From patchwork Fri Sep 15 15:08:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 723115 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp435270wro; Fri, 15 Sep 2023 08:09:45 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFe/SuIG2EpuxkN+GRFpUBejGYtWNzFBhUW+dpl9CWEgpJo3bfunyzMJO7RQVf0a4oSN1s3 X-Received: by 2002:a0c:e414:0:b0:651:69d9:1523 with SMTP id o20-20020a0ce414000000b0065169d91523mr2116016qvl.16.1694790585548; Fri, 15 Sep 2023 08:09:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694790585; cv=none; d=google.com; s=arc-20160816; b=h5+sV62xF7WttjR/WtqcltlvtPxMZbsHvoIL/at3zNrgjIvpcf2VDxUtMz9H45X6SD Xj1IJX3Isy2iRrgK8SXVV9lXR6Jqb+Qc6ULLvvhl/BUHBmSsoihxOwVMgzawtC3mjnrn hikNo06na2vl6Ney8egyF3TCKhiCt3nChX5vz1As50cGYks20LJ3/XmILm/Ts2uywkCu KhA93+9XHv1UC80fvqJFmiW5EFzreShWEe3sydaZoZb/DJIwPaI/SL3D+MLM5Ntj9BPb G/SzAayOV6N50Er8F41158gkmaU6A+Iys6205A72uQORQRl/E59E0Na855Rd92ZsZOQq kqJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=CcvLekfgkgzF278cqTU4MlqaFKkEi2GbfnC328zBuE0=; fh=DsCgdbJtKECnOY1fjYjo4d9wfAF6VXeyryDhRNU8UH0=; b=RX6nJxrM8zCpCIikRGLzhIsHhqnY2lRyNr17+9YATzpHA2uciqi0uNFBkKcxyK2zEh Cvq7fN1onW0Z1QLyqvfrJIS4+0sgNofQI3zH7JBeqaOdRHq+X5eMgYrqWQJHpquIczYi MTvYdHVyp1p3zt8j9Hu66g+BcKtX4iqwsMltF39Kc+H0/6zLD9SEL4d9We0SLePotk6P wd0zyS1huJGxPLxH22/DgrTYnyaWxLOZOf8Wt9+ahIj6DlPmISOg8ckM7IeH7yESCGGd L9LMqPE5dQCiu2gy6WNRQEtjGyx700yvYojDv/CpKO7qnc+rBdy3WVFPGRHdM3F33obI Trnw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=wtYI5RmU; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id p4-20020a0cf544000000b006563b57c433si533939qvm.160.2023.09.15.08.09.45 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 08:09:45 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=wtYI5RmU; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhARU-0003P5-W2; Fri, 15 Sep 2023 11:09:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhARH-0003F0-J9 for qemu-devel@nongnu.org; Fri, 15 Sep 2023 11:08:59 -0400 Received: from mail-wr1-x42a.google.com ([2a00:1450:4864:20::42a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhARF-00029o-LW for qemu-devel@nongnu.org; Fri, 15 Sep 2023 11:08:59 -0400 Received: by mail-wr1-x42a.google.com with SMTP id ffacd0b85a97d-313e742a787so1485615f8f.1 for ; Fri, 15 Sep 2023 08:08:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694790536; x=1695395336; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CcvLekfgkgzF278cqTU4MlqaFKkEi2GbfnC328zBuE0=; b=wtYI5RmU0grbK63HB4S1Uxo0NrXI+c/Xu0ig2rWdRkoNjvZkSQ7RqU2PNqRS1+Tn2N +NQz0QLq06Y4UZBrKie10r/rvtYTdfHcpVtWfSrgFnBSJaNlCh3rPBYAUHNgeoE0V1C3 V6vNxg3vdfyy0QKSG3PggGP/lt/T9ZKURIH/e7yovRW4630CL/phJE+Cqm+eZzUP7tLL styrDqGnRSXKcYK0UH9Jb81Vk3CLNiC74LJYyzrrtg1yjRD2TTZY8MLWvSVfXwnw3Bi+ UgqydRx/8FSx133hYP4p9oQG8JvsVLOQs8wdnOwuga4sZq8Nnu37pH55hI+RTp4tEpQE IC0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694790536; x=1695395336; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CcvLekfgkgzF278cqTU4MlqaFKkEi2GbfnC328zBuE0=; b=XviFIel2H9cxkfZPgfB9TwDVcD7JxxGYv0jvlTjlBM1nDKR4g7sUoDiJQQuBTqVVYA Bwh+w8YS7NGJ0VPQONf+KltR2Z2X/6Tw6/jNsY0mZC8827fWqz/kobGtLpS5V5R86Yyg BQXnAHaI07JiM/Gnr4sJsQldIfqkyNhVgtu+cpBW0OyviTzF8Sgow+MLWjBnjCACGqby ynxpUm+fzrc3LyWObeBis5GOop5UKHXG1EuZqy6iViaCDbatj5ibJukGT/cDgS0dsAa5 yH9dqQmuzI3YbI0kKOQUPYT6UTLlOn/AJeVU1e4euFmiIhAmJsNUM/kJepMjKFnwy7v0 iyxQ== X-Gm-Message-State: AOJu0YwOwhKH1iOMS7DbO4Ks4SubVWuY9U6OiPh7pZy60jF25uipNw4J yDKalMruicMHi1sIWSJp+g5VMA== X-Received: by 2002:a5d:6585:0:b0:317:dd94:d38b with SMTP id q5-20020a5d6585000000b00317dd94d38bmr4283710wru.10.1694790535961; Fri, 15 Sep 2023 08:08:55 -0700 (PDT) Received: from zen.linaroharston ([85.9.250.243]) by smtp.gmail.com with ESMTPSA id z25-20020a1c4c19000000b003fe29f6b61bsm4832634wmf.46.2023.09.15.08.08.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 08:08:53 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id C44E81FFBB; Fri, 15 Sep 2023 16:08:50 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Cc: Nicholas Piggin , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?Phil?= =?utf-8?q?ippe_Mathieu-Daud=C3=A9?= , John Snow , =?utf-8?q?Alex_Benn=C3=A9e?= , Cleber Rosa , Beraldo Leal , Wainer dos Santos Moschetta Subject: [PULL 8/9] tests/avocado: Fix console data loss Date: Fri, 15 Sep 2023 16:08:48 +0100 Message-Id: <20230915150849.595896-9-alex.bennee@linaro.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230915150849.595896-1-alex.bennee@linaro.org> References: <20230915150849.595896-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::42a; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org From: Nicholas Piggin Occasionally some avocado tests will fail waiting for console line despite the machine running correctly. Console data goes missing, as can be seen in the console log. This is due to _console_interaction calling makefile() on the console socket each time it is invoked, which must be losing old buffer contents when going out of scope. It is not enough to makefile() with buffered=0. That helps significantly but data loss is still possible. My guess is that readline() has a line buffer even when the file is in unbuffered mode, that can eat data. Fix this by providing a console file that persists for the life of the console. Signed-off-by: Nicholas Piggin Reviewed-by: "Daniel P. Berrangé" Message-Id: <20230912131340.405619-1-npiggin@gmail.com> Reviewed-by: Philippe Mathieu-Daudé Acked-by: John Snow Signed-off-by: Alex Bennée Message-Id: <20230914155422.426639-9-alex.bennee@linaro.org> diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index c16a0b6fed..35d5a672db 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -191,6 +191,7 @@ def __init__(self, self.sock_dir, f"{self._name}.con" ) self._console_socket: Optional[socket.socket] = None + self._console_file: Optional[socket.SocketIO] = None self._remove_files: List[str] = [] self._user_killed = False self._quit_issued = False @@ -509,6 +510,11 @@ def _early_cleanup(self) -> None: # If we keep the console socket open, we may deadlock waiting # for QEMU to exit, while QEMU is waiting for the socket to # become writable. + if self._console_file is not None: + LOG.debug("Closing console file") + self._console_file.close() + self._console_file = None + if self._console_socket is not None: LOG.debug("Closing console socket") self._console_socket.close() @@ -874,12 +880,25 @@ def console_socket(self) -> socket.socket: Returns a socket connected to the console """ if self._console_socket is None: + LOG.debug("Opening console socket") self._console_socket = console_socket.ConsoleSocket( self._console_address, file=self._console_log_path, drain=self._drain_console) return self._console_socket + @property + def console_file(self) -> socket.SocketIO: + """ + Returns a file associated with the console socket + """ + if self._console_file is None: + LOG.debug("Opening console file") + self._console_file = self.console_socket.makefile(mode='rb', + buffering=0, + encoding='utf-8') + return self._console_file + @property def temp_dir(self) -> str: """ diff --git a/tests/avocado/avocado_qemu/__init__.py b/tests/avocado/avocado_qemu/__init__.py index 33090903f1..0172a359b7 100644 --- a/tests/avocado/avocado_qemu/__init__.py +++ b/tests/avocado/avocado_qemu/__init__.py @@ -137,7 +137,7 @@ def _console_interaction(test, success_message, failure_message, assert not keep_sending or send_string if vm is None: vm = test.vm - console = vm.console_socket.makefile(mode='rb', encoding='utf-8') + console = vm.console_file console_logger = logging.getLogger('console') while True: if send_string: