pfsense 24.11 support for the api and/or RestAPI

[berwegera]

Hello everyone
I think I was a bit hasty with the installation of pfsense 24.11 .... in any case I can't get API version 1.7.6 or RestAPI version 2.2.2 to work. I have tried both versions with Authentication Mode = API Token. I always get the following json in node red return {‘status’: ‘forbidden’, ‘code’:403, ‘return’:4, ‘message’: ‘Authorisation failed’, ‘data’:[]}.

If I switch back to pfsense 2.7.2 then version 1.7.6 runs without any problems!!!
Can anyone help me? Has anyone managed to get the Api or RestAPI (version? Authentication Mode?) running under Pfsense 24.11?

Greetings and thanks for any help
Andi

[jaredhendrickson13]

Support for 24.11 is being worked on, but I don't anticipate it anytime soon. There were many changes in 24.11 that have broken various parts of the package and without a CE equivalent or development snapshots it becomes very difficult to debug effectively. For now I'd recommend staying on Plus 24.03 or CE 2.7.2.

[berwegera]

That works for me. I will return to 24.03. 24.11 has no changes that I couldn't do without. It is more important to me to be able to control the firewall via the API.
Thank you very much for the quick reply. I am looking forward to the new version of pfesne-api.

[madbrain76]

Does anyone know how to roll back to 24.03 ? I have config backups, but not the image.

[sonmezcem]

I had a similar situation. I didn't roll back the older version but I have managed to restore config backups to the same version of Pfsense.

So in my opinion following should work;

• Take the current image backup. (just in case to roll back)
• Note all the installed extra packages.
• Than install Pfsense 24.03
• Install extra packages that you took the note on step 2
• Then restore your config backups.
• If needed reload configurations for extra packages.

Do it, and see if it works for you...

[berwegera]

I did it with the netgate installer, equivalent to a new installation + then reloaded the configuration (the backup) from 24.11 (is compatible with the 24.03 version). I had to go this way because I had no ZFS installation and came directly from the 2.7.2 CE edition. The netgate installer has no version ‘included’ but during the installation process (requires internet connection) the version 24.03 or 24.11 can be selected and it then downloads and installs the desired version.
Preferable and faster and easier is of course the ZFS way ... -> see your next post :)

Translated with DeepL.com (free version)

[madbrain76]

@sonmezcem Thank you ! Someone on the Netgate forum suggested I rollback using System -> boot environments. If installed on ZFS, it will show snapshots, which can be restored. I was able to rollback to 24.03 very painlessly that way. pfSense API works again, too. Very glad I used ZFS.

[jaredhendrickson13]

Dev builds with very basic compatibility with 24.11 are available in this pre-release. This should address some of the issues seen with authentication and configuration access; and allow the package to work on a fundamental basis. There are still issues I am actively working to fix, and I'm sure there are more that I'm unable to see with my limited access to pfSense Plus. For those able to test, especially on Netgate hardware, it would be much appreciated and definitely speed up the process.

[madbrain76]

@jaredhendrickson13 ,

First time I tried it, I not able to install it .

Fetching pfSense-24.11-pkg-RESTAPI.pkg: .......... done
Installing pfSense-pkg-RESTAPI-2.3_0-dev-96ab4c0...
the most recent version of pfSense-pkg-RESTAPI-2.2_2 is already installed

I had to remove the previous package with "pkg delete -y pfSense-pkg-RESTAPI-2.2_2" .

I then installed it with
pkg-static -C /dev/null add https://github.com/jaredhendrickson13/pfsense-api/releases/download/v2.3.0-dev-96ab4c0/pfSense-24.11-pkg-RESTAPI.pkg

However, system -> API produces a 403 forbidden. I see that there is now a System -> Rest API menu instead.

I don't see the place to create a client ID and token. It's admittedly been a while. I think I need to revert to the previous 24.03 snapshot once again to see what it looks like. I did all this setup in late May originally and have completely forgotten already. A quick glance at the doc did not help. I also did not write any of the client code I used - ChatGPT did it at my direction, because I'm just not good enough with Python. Good thing GPT heavily commented the source, though.

[jaredhendrickson13]

The script you provided looks to be for v1, that script would not work on v2. You can see for sure if the URL you're using has v1 in it as opposed to v2. You may have both the v1 and v2 packages installed, but this looks like you're only using v1.

Dev build for v1 on 24.11:
https://github.com/jaredhendrickson13/pfsense-api/releases/download/v1.8.0-dev-aa40a2c/pfSense-24.11-pkg-API.pkg

Dev build for v2 on 24.11:
https://github.com/jaredhendrickson13/pfsense-api/releases/download/v2.3.0-dev-96ab4c0/pfSense-24.11-pkg-RESTAPI.pkg

[madbrain76]

Thanks for your response !
The pfSense URL I'm using with my script does not have either v1 or v2. It is as follows :
PF_DHCPD_IF_URL="https://pfsense.localdomain/api/v1/services/dhcpd/static_mapping?interface=LAN"

I was wrong about using the 2.2 package previously. I'm booted up with 24.03 right now with the script working, and pkg info | grep -i api shows that I'm using
pfSense-pkg-API-1.7_3 pfSense API package

I'm going to try again with the 1.8 package in 24.11 .

[madbrain76]

I'm unable to install the 1.8 package on pfSense 24.11 .

Fetching pfSense-24.11-pkg-API.pkg: .......... done
Installing pfSense-pkg-API-1.8_0-dev-aa40a2c...
pkg-static: wrong architecture: FreeBSD:14:* instead of FreeBSD:15:amd64

Failed to install the following 1 package(s): https://github.com/jaredhendrickson13/pfsense-api/releases/download/v1.8.0-dev-aa40a2c/pfSense-24.11-pkg-API.pkg

[jaredhendrickson13]

I had the build version for 24.11 mixed up in the pipeline so it built on the wrong FreeBSD version, sorry about that. I just rebuilt on the correct version and updated the release. Try again.

[madbrain76]

Thanks ! I was able to install it this time. However, I couldn't get my script to work. I modified it to add more logging, and used a diffferent URL, since the old is no longer valid under 24.11 . The request got a 301, "Moved".

# -*- coding: utf-8 -*-
import sys
import json
import subprocess
import os
import logging

# Set up logging configuration with UTF-8 encoding
logging.basicConfig(
    filename="script_errors.log",  # Log to a file named 'script_errors.log'
    level=logging.ERROR,  # Log only errors
    format="%(asctime)s - %(levelname)s - %(message)s",
    encoding="utf-8"  # Ensure UTF-8 encoding for the log file
)

# Define environment variable names
client_id = os.getenv("PF_CLIENT_ID")  # Environment variable for Client ID
token = os.getenv("PF_TOKEN")         # Environment variable for Token
pfSense_url = os.getenv("PF_DHCPD_IF_URL")  # Environment variable for pfSense URL

# Check if credentials and URL are present
if not client_id or not token or not pfSense_url:
    print("Error: Missing environment variables. Please set PF_CLIENT_ID, PF_TOKEN, and PF_DHCPD_IF_URL.")
    logging.error("Missing environment variables: PF_CLIENT_ID, PF_TOKEN, or PF_DHCPD_IF_URL")
    sys.exit(1)

# Construct the Authorization header
auth_header = f"{client_id} {token}"

# Define curl command with Authorization header and capture output
curl_command = ["curl", "-k", "-X", "GET", "-H", f"Authorization: {auth_header}", "-w", "%{http_code}", "-o", "/tmp/curl_output.txt", pfSense_url]

try:
    # Execute curl command and capture output, including HTTP status code
    process = subprocess.run(
        curl_command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, encoding='utf-8'
    )

    # Extract HTTP status code from curl's output (last part from -w "%{http_code}")
    http_status_code = process.stdout.strip().split("\n")[-1]

    # Check for HTTP error (non-200 status code)
    if not http_status_code.startswith("2"):  # Not 2xx range
        print(f"HTTP Error: Received status code {http_status_code}")
        logging.error(f"HTTP Error: Received status code {http_status_code}")
        sys.exit(1)

    # Extract response (assuming JSON format)
    with open("/tmp/curl_output.txt", "r", encoding="utf-8") as file:
        dhcp_static_mappings_json = file.read()

except Exception as e:
    print(f"Error fetching DHCP data: {e}")
    logging.error(f"Error fetching DHCP data: {e}")
    sys.exit(1)

try:
    # Parse the JSON response
    dhcp_static_mappings = json.loads(dhcp_static_mappings_json)

    # Print raw JSON mappings on successful retrieval
    print(dhcp_static_mappings_json)

except json.JSONDecodeError as e:
    # Log the JSON that couldn't be parsed and the error
    print(f"Error parsing JSON response: {e}")
    logging.error(f"Error parsing JSON response: {e}")
    logging.error(f"Raw JSON that failed to parse: {dhcp_static_mappings_json}")
    sys.exit(1)

The URL is used was
https://pfsense.localdomain/services_dhcp.php

Unfortunately, the result was :

Error parsing JSON response: Expecting value: line 1 column 1 (char 0)

script_errors.log shows that the data fetched was an HTML page, rather JSON.

[berwegera]

It works for me I have updated again to the pfsense 24.11 version. -> Api was then no longer available -> Then installed the new pfSense API v1.8.0-dev-aa40a2c -> Assigned new API keys -> and voila, it works for my needs. Authentication works and I switch a few rules on and off using the endpoints /api/v1/firewall/rule with {‘disabled’: false, ‘tracker’: ‘0100000101’, ‘apply’: true } etc.

Thanks for the quick solution!!!

[madbrain76]

Does anyone know what endpoint I should use in 24.11 to get the list of DHCP reservations for the LAN interface ?
Until 24.03, I used https://pfsense.localdomain/api/v1/services/dhcpd/static_mapping?interface=LAN . That URL no longer works in 24.11 .

[jaredhendrickson13]

That is still the correct URL for v1 on 24.11. It seems to still be present and functional for me on 24.11 at least. Did you run /etc/rc.restart_webgui after installing the package? What do you receive when you make a request to that URL?

[madbrain76]

It's working now. Not sure what I was doing wrong earlier, but glad it works.

[jaredhendrickson13]

I believe all the apparent issues have been addressed and v1.8.0 and v2.3.0 have been officially released. There are still some cosmetic issues with some buttons in the UI pages, but there's not a great way to address those without breaking them for CE 2.7.2 and Plus 24.03. Those issues should be addressable once a new CE version is released. I've created #620 to track that issue.

Feel free to open issues if you have any issues.