Merge branch 'main' into build

README.md

@@ -10,7 +10,7 @@ The proxy works in the background with a menu bar/taskbar helper or as a headles
 
 ### Example use-cases<a id="example-use-cases"></a>
 - You need to use an Office 365 email account, but don't get on with Outlook.
-The email client you like doesn't support OAuth 2.0, which became mandatory [in January 2023](https://techcommunity.microsoft.com/t5/exchange-team-blog/basic-authentication-deprecation-in-exchange-online-september/ba-p/3609437).
+The email client you like doesn't support OAuth 2.0, which became mandatory [in January 2023](https://techcommunity.microsoft.com/t5/exchange-team-blog/basic-authentication-deprecation-in-exchange-online-september/ba-p/3609437) ([September 2025 for SMTP](https://techcommunity.microsoft.com/t5/exchange-team-blog/exchange-online-to-retire-basic-auth-for-client-submission-smtp/ba-p/4114750)).
 - You used to use Gmail via IMAP/POP/SMTP with your raw account credentials (i.e., your real password), but cannot do this now that Google has disabled this method, and don't want to use an [App Password](https://support.google.com/accounts/answer/185833) (or cannot enable this option).
 - You have an account already set up in an email client, and you need to switch it to OAuth 2.0 authentication.
 You can edit the server details, but the client forces you to delete and re-add the account to enable OAuth 2.0, and you don't want to do this.
@@ -61,7 +61,7 @@ It will notify you if this is the case.
 As part of the proxy setup process you need to provide an OAuth 2.0 `client_id` and `client_secret` to allow it to authenticate with email servers on your behalf.
 
 If you have an existing client ID and secret for a desktop app, you can use these directly in the proxy.
-If this is not possible, you can also reuse the client ID and secret from any email client that supports IMAP/POP/SMTP OAuth 2.0 authentication with the email server you would like to connect to (such as [the](https://github.com/mozilla/releases-comm-central/blob/812b7c9068ca5cac0580b0ddbea8e34c141cd441/mailnews/base/src/OAuth2Providers.jsm) [various](https://github.com/Foundry376/Mailspring/blob/master/app/internal_packages/onboarding/lib/onboarding-constants.ts) [open](https://gitlab.gnome.org/GNOME/evolution-data-server/-/blob/master/CMakeLists.txt) [source](https://gitlab.gnome.org/GNOME/gnome-online-accounts/-/blob/master/meson_options.txt) [clients](https://github.com/M66B/FairEmail/blob/master/app/src/main/res/xml/providers.xml) with OAuth 2.0 support), but please do this with care and restraint as access through reused tokens will be associated with the token owner rather than your own client.
+If this is not possible, you can also reuse the client ID and secret from any email client that supports IMAP/POP/SMTP OAuth 2.0 authentication with the email server you would like to connect to (such as [the](https://github.com/mozilla/releases-comm-central/blob/812b7c9068ca5cac0580b0ddbea8e34c141cd441/mailnews/base/src/OAuth2Providers.jsm) [many](https://github.com/mozilla/releases-comm-central/blob/master/mailnews/base/src/OAuth2Providers.sys.mjs) [existing](https://github.com/Foundry376/Mailspring/blob/master/app/internal_packages/onboarding/lib/onboarding-constants.ts) [open](https://gitlab.gnome.org/GNOME/evolution-data-server/-/blob/master/CMakeLists.txt) [source](https://gitlab.gnome.org/GNOME/gnome-online-accounts/-/blob/master/meson_options.txt) [clients](https://github.com/M66B/FairEmail/blob/master/app/src/main/res/xml/providers.xml) with OAuth 2.0 support), but please do this with care and restraint as access through reused tokens will be associated with the token owner rather than your own client.
 
 If you do not have access to credentials for an existing client you will need to register your own.
 The process to do this is different for each provider, but the registration guides for several common ones are linked here.
@@ -137,7 +137,7 @@ If your network requires connections to use an existing proxy, you can instruct
 After installing its requirements, the proxy script can be packaged as a single self-contained executable using [pyinstaller](https://pyinstaller.org/) if desired: `pyinstaller --onefile emailproxy.py`.
 If you are using the GUI version of the proxy, you may need to add `--hidden-import timeago.locales.en_short` until [this `timeago` issue](https://github.com/hustcc/timeago/issues/40) is resolved.
 
-Python 3.6 or later is required to run the proxy.
+Python 3.7 or later is required to run the proxy.
 The [python2 branch](https://github.com/simonrob/email-oauth2-proxy/tree/python2) provides minimal compatibility with python 2.7, but with a limited feature set, and no ongoing maintenance.
 See [issue 38](https://github.com/simonrob/email-oauth2-proxy/issues/38) for further discussion.
 
@@ -181,6 +181,7 @@ It is often helpful to be able to view the raw connection details when debugging
 This can be achieved using `telnet`, [PuTTY](https://www.chiark.greenend.org.uk/~sgtatham/putty/) or similar.
 For example, to test the Office 365 IMAP server from the [example configuration](https://github.com/simonrob/email-oauth2-proxy/blob/main/emailproxy.config), first open a connection using `telnet 127.0.0.1 1993`, and then send a login command: `a1 login e@mail.com password`, replacing `e@mail.com` with your email address, and `password` with any value you like during testing (see above for why the password is irrelevant).
 If you have already authorised your account with the proxy you should see a response starting with `a1 OK`; if not, this command should trigger a notification from the proxy about authorising your account.
+Note that POP and SMTP are different protocols, and while they can be tested in this way, they require different commands to be sent – see [this issue comment](https://github.com/simonrob/email-oauth2-proxy/issues/251#issuecomment-2133976839) for further details.
 
 If you are using a [secure local connection](https://github.com/simonrob/email-oauth2-proxy/blob/main/emailproxy.config) the interaction with the remote email server is the same as above, but you will need to use a local debugging tool that supports encryption.
 The easiest approach here is to use [OpenSSL](https://www.openssl.org/): `openssl s_client -crlf -connect 127.0.0.1:1993`.

emailproxy.config

@@ -3,7 +3,11 @@ documentation = This is a sample Email OAuth 2.0 Proxy configuration file. Confi
 	[Server setup] and [Account setup] sections below. You may delete any servers or accounts that you do not intend to
 	use. Documentation is provided inline, with example setups for Gmail and Office 365 (though you will need to enter
 	your own desktop app API client credentials in the accounts section). Use the `Reload configuration file` menu
-	option or send a SIGHUP signal (or restart the proxy) to apply any changes.
+	option or send a SIGHUP signal (or quit the proxy before editing, then restart) to apply any changes.
+format = This file's format is documented at docs.python.org/library/configparser#supported-ini-file-structure. Values
+	that span multiple lines should be indented deeper than the first line of their key (as in this comment). Quoting
+	of values is not required. Documentation sections can be removed if needed (though it is advisable to leave these
+	in place for reference) - thw only required sections are the individual server and account items of your setup.
 warning = Do not commit changes to this file into a public repository (e.g., GitHub, etc). While the proxy encrypts the
 	OAuth 2.0 tokens it obtains and saves on your behalf, it cannot protect these against offline brute-force attacks.
 
@@ -55,7 +59,7 @@ server_port = 995
 local_address = 127.0.0.1
 
 [SMTP-1587]
-server_address = smtp.office365.com
+server_address = smtp-mail.outlook.com
 server_port = 587
 server_starttls = True
 local_address = 127.0.0.1
@@ -79,9 +83,9 @@ local_address = 127.0.0.1
 [Account setup]
 documentation = Accounts are specified using your email address as the section heading (e.g., [your.email@gmail.com],
 	etc, below). Account usernames (i.e., email addresses) must be unique - only one entry per account is permitted.
-	Each account section must provide values for `permission_url`, `token_url`, `oauth2_scope` and `redirect_uri`. If
-	you are adding an account for a service other than the examples shown below then the provider's documentation should
-	provide these details.
+	Each account section must provide values for at least `token_url`, `oauth2_scope` and `client_id`. Depending on the
+	OAuth 2.0 flow you are using, other values may also be required (see examples below). If you are adding an account
+	for a service other than the examples shown below then the provider's documentation should provide these details.
 
 	You will also need to add your own `client_id` and `client_secret` values as indicated below. These can either be
 	reused from an existing source (such as another email client that supports OAuth 2.0), or you can register and use
@@ -109,6 +113,13 @@ documentation = Accounts are specified using your email address as the section h
 	in the example below) in order to allow the proxy to refresh its access token on your behalf. The proxy will still
 	work if this parameter is not included, but you will need to re-authenticate extremely often (about once per hour).
 
+	- The example Office 365 configuration entries below use an OAuth 2.0 scope that clearly specifies IMAP, POP and
+	SMTP permission. If you do not require one or more of these protocols, you may remove the relevant values to ensure
+	the access tokens obtained on your behalf are as precisely-targeted as possible. Conversely, it is also possible to
+	replace these specific scopes with the more generic `https://outlook.office365.com/.default`. Switching to a broader
+	scope value may also be needed if you are using a version of O365 delivered by a regional provider (e.g., 21Vianet).
+	See: https://github.com/simonrob/email-oauth2-proxy/issues/255 for more details and discussion.
+
 	- By default, new Entra (Azure AD) clients are accessible only within your own tenant. If you are registering a new
 	client to use with the proxy (and do not want to make it available outside your own organisation) you will need to
 	replace `common` with your tenant ID in the Office 365 `permission_url` and `token_url` values below. Alternatively,
@@ -164,9 +175,9 @@ documentation = Accounts are specified using your email address as the section h
 
 	Advanced account configuration:
 	- For most configurations the default `redirect_uri` value of `http://localhost` is correct, unless you have
-	explicitly set your OAuth 2.0 client to use a different address here (which you will need to manually redirect to
-	the proxy). Please also note that this address is `http` and not `https` (which is not supported unless you provide
-	a secure connection yourself outside of the proxy using an external tool).
+	explicitly set the OAuth 2.0 client configuration with your provider to use a different address for this purpose
+	(e.g., redirecting via an external domain). If this is the case, you will need to manually redirect this to the
+	proxy. Please note that in most cases the address is indeed `http://localhost`, not `https`.
 
 	- When using the `--local-server-auth` option you will need to either run the proxy with additional privileges to
 	use the implicit default port 80 (e.g., via sudo) or specify a different port (and/or a different host if needed) -
@@ -183,7 +194,7 @@ documentation = Accounts are specified using your email address as the section h
 [your.office365.address@example.com]
 permission_url = https://login.microsoftonline.com/common/oauth2/v2.0/authorize
 token_url = https://login.microsoftonline.com/common/oauth2/v2.0/token
-oauth2_scope = https://outlook.office365.com/IMAP.AccessAsUser.All https://outlook.office365.com/POP.AccessAsUser.All https://outlook.office365.com/SMTP.Send offline_access
+oauth2_scope = https://outlook.office.com/IMAP.AccessAsUser.All https://outlook.office.com/POP.AccessAsUser.All https://outlook.office.com/SMTP.Send offline_access
 redirect_uri = http://localhost
 client_id = *** your client id here ***
 client_secret = *** your client secret here ***
@@ -217,7 +228,6 @@ documentation = *** note: this is an advanced O365 account example; in most case
 token_url = https://login.microsoftonline.com/*** your tenant id here ***/oauth2/v2.0/token
 oauth2_scope = https://outlook.office365.com/.default
 oauth2_flow = client_credentials
-redirect_uri = http://localhost
 client_id = *** your client id here ***
 client_secret = *** your client secret here ***
 
@@ -226,7 +236,6 @@ documentation = *** note: this is an advanced O365 account example; in most case
 token_url = https://login.microsoftonline.com/*** your tenant id here ***/oauth2/v2.0/token
 oauth2_scope = https://outlook.office365.com/IMAP.AccessAsUser.All https://outlook.office365.com/POP.AccessAsUser.All https://outlook.office365.com/SMTP.Send offline_access
 oauth2_flow = password
-redirect_uri = http://localhost
 client_id = *** your client id here ***
 client_secret = *** your client secret here ***
 
@@ -235,7 +244,6 @@ documentation = *** note: this is an advanced Google account example; in most ca
 token_url = https://oauth2.googleapis.com/token
 oauth2_scope = https://mail.google.com/
 oauth2_flow = service_account
-redirect_uri = http://localhost
 client_id = file
 client_secret = *** your /path/to/service-account-key.json here ***
 
@@ -244,7 +252,6 @@ documentation = *** note: this is an advanced Google account example; in most ca
 token_url = https://oauth2.googleapis.com/token
 oauth2_scope = https://mail.google.com/
 oauth2_flow = service_account
-redirect_uri = http://localhost
 client_id = key
 client_secret = *** your pasted service account JSON key file contents here,
 	making sure to indent all lines by at least one space ***
@@ -281,6 +288,15 @@ documentation = The parameters below control advanced options for the proxy. In
 	using catch-all accounts or the proxy's `--cache-store` parameter you must manually remove unencrypted secrets from
 	the local configuration file after the encrypted secret has been created (i.e., this will not be automatic).
 
+	- use_login_password_as_client_credentials_secret (default = False): When using the O365 client credentials grant
+	(CCG) flow, rather than encrypting the client secret (see above), the proxy can be instructed to use the given
+	IMAP/POP/SMTP login password as the client secret. This approach removes the risk of storing the unencrypted client
+	secret in the proxy's configuration file, and also means there is no risk of unauthorised account access when using
+	the O365 CCG flow in conjunction with the proxy's catch-all mode (see below). To enable this option, set
+	`use_login_password_as_client_credentials_secret` to True. Note that if a `client_secret` value is present in your
+	account's configuration entry, that value will be used instead of the given IMAP/POP/SMTP login password even if
+	this option is enabled. To avoid this, remove the entire `client_secret` line from the configuration entry.
+
 	- allow_catch_all_accounts (default = False): The default behaviour of the proxy is to require a full separate
 	configuration file entry for each account. However, when proxying multiple accounts from the same domain it can be
 	cumbersome to have to create multiple near-identical configuration profiles. To simplify this the proxy supports
@@ -297,4 +313,5 @@ documentation = The parameters below control advanced options for the proxy. In
 [emailproxy]
 delete_account_token_on_password_error = True
 encrypt_client_secret_on_first_use = False
+use_login_password_as_client_credentials_secret = False
 allow_catch_all_accounts = False