Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send to API and Test code mapping #8

Open
ubeffiong opened this issue Dec 4, 2022 · 13 comments · May be fixed by #9
Open

Send to API and Test code mapping #8

ubeffiong opened this issue Dec 4, 2022 · 13 comments · May be fixed by #9
Assignees
Labels
enhancement New feature or request

Comments

@ubeffiong
Copy link

ubeffiong commented Dec 4, 2022

Thanks for being generous 😊

please can you add the following functionalities;

  1. to get orders orders from API,
  2. send acknowledgement (true) after receiving the orders.
  3. Able to request for orders after getting a order ID (read by machine barcode).
  4. Able to post results to API (using order ID).
  5. Finally functionality to map host test code to machine test code if they are different.
  6. Thanks you so much
@roy-harmon
Copy link
Owner

Thanks for being generous 😊

please can you add the following functionalities;

  1. to get orders orders from API,

Can you please elaborate on this? What kind of API did you have in mind, specifically? That is, are you talking about a web API, or something like a software library?

  1. send acknowledgement (true) after receiving the orders.

Currently, acknowledgement is sent as required by the specification. This isn't going to change, but if you're talking about it in regards to your API suggestion, then this will depend on what kind of API we're talking about. If it's a REST API, then I would expect acknowledgement to come in the form of a response status code. Is that what you had in mind?

  1. Able to request for orders after getting a order ID (read by machine barcode).

I'm afraid I don't understand what you're asking for here. Can you please explain it a little more? I'm thinking the machine would query the LIS for test orders after scanning the barcode, but that functionality should already be working.

  1. Able to post results to API (using order ID).

This could get complicated, depending on what you have in mind. Are you talking about something like webhooks, or do you just want it to send the order ID's test results via HTTP POST request to a predetermined URI whenever a new result is available?

  1. Finally functionality to map host test code to machine test code if they are different.

This is something that I would have expected to be handled on the LIMS side, but I can see a use case for this. If there was an option to have it query a mapping table in the database for the test code, would that suffice?

  1. Thanks you so much

You're welcome! I haven't found much time to devote to this project since I started a new job earlier in the year, but I haven't forgotten about it and I still have plans for its development. Since I mostly use Java in my work these days, I've been considering migrating this project to Java. It's very similar to the C# that UniversaLIS is currently written in, and it could further improve accessibility by using a more open language. More importantly, it would let me narrow my focus and use the same language and skills for this project as I use for my day job.

@ubeffiong
Copy link
Author

  1. I have a web application (LIMS) that has a getorder REST API for LIS middleware integration with laboratory machines. I'm looking at UniversaLIS consuming the data from it (patients orders) rather than pulling the data from using SQL queries from the database as you provided.

2.Yes, it is a REST API. The reply to the API is needed to ensure that orders are not sent repeatedly to UniversaLIS.

  1. Yes the machine query's the LIS middleware (UniversaLIS) for an order after scanning the barcode. However, since we are not looking at saving the orders from the REST API in a database, UniversaLIS will need to request the orders from the API at every time the machine makes a request.

  2. Yes I'm looking at sending the results to a HTTP post request (LIMS)

  3. Since we are looking at eliminating the database, I'm thinking of having a file to handle the test mapping. This will make it easy to adopt UniversaLIS implementation for various machines with similar integration without going to the database to create mapping for every machine test code.

  4. Please don't discontinue modification and support for UniversaLIS C# 🥺. I'm new to C# I have learnt alot studying how you did the implementation, the approach is very simple. I intend to grow speedily. I'm very grateful.

(Sorry for unstructured reply)

@roy-harmon
Copy link
Owner

All right, that answers those questions, and now your requests make a lot of sense. I'm sorry it took me a while to catch on, but I've been out of the lab for over a year now and had to readjust my perspective.
I want to make sure I follow your desired work flow, so if I've misunderstood something, please correct me.

First, the machine scans the specimen barcode and queries the LIS for orders. The LIS then sends a GET request to the LIMS, which responds with the test orders (including the LIMS test code) for that specimen. The LIS then checks the list of test code mappings for the machine that sent the original query, and if any of them match the orders from the LIMS, then those are sent to the machine; otherwise, the LIS tells the machine that there are no orders for the specified barcode.

Assuming that there were some matching orders, the machine completes the tests and transmits the results to the LIS. The LIS then sends a POST request to the LIMS, and upon receiving a 2xx response, it sends confirmation to the machine that the result has been received/accepted.

Is that what you have in mind, or have I missed something? It sounds like the following configuration options will be needed:

  • A setting for an operational mode in which the LIS serves only as a mediator between the instrumentation and the LIMS, without accessing any database. This seems like a great idea.
  • Settings for each instrument's test list and any test code mappings. There are a few ways to do this and I'll have to think about what would be the best option. I'm thinking it might be good to scan the mapping files on startup and store the data in some kind of data structure so it doesn't have to access text files for each request. The mapping files would presumably be YAML, but I'll need to refresh my memory before I can say more about the formatting.
  • Separate text fields for the API endpoints, since the URI for getting orders probably won't be the same as the one for posting results. This should be fairly straightforward.

Can you tell me anything about the API specification of your LIMS software?

@ubeffiong
Copy link
Author

ubeffiong commented Dec 5, 2022

Thank you very much.

• You are absolutely correct.
• However, I failed to add that it's not everytime the machine will need to ask LIS for a particular order. Machines like Abbott has the ability of saving orders on its worklist. So the LIS query and requery's the LMIS (let's say every seconds) for orders and then forward to the machine. So a machine like Abbott can receive those orders on a worklist. And will still have the ability to request for any samples_id (barcode) not found on the worklist. However, machines like Cobas do not save orders on a worklist rather each time the barcode is scanned a request is sent to the LIS to provide the orders.

• About the API specification.
Format - presented as JSON object, filed name maybe case sensitive. (but I can handle that if need be to modify)

Data Structure and Type- Data type are mostly string data (I can make changes to this if need be)

@ubeffiong
Copy link
Author

@roy-harmon greetings!!!
Sorry to bother you, I haven't heard from you since our last discussion.
Please I would like to know the feedback.

Thanks

@roy-harmon
Copy link
Owner

Sorry, I'm going to need some time to think about how to implement this. I'm having a very busy week.

@ubeffiong
Copy link
Author

Alright Sir. I really appreciate

@roy-harmon
Copy link
Owner

I'm thinking each instrument's configuration will need a setting for whether it can save new orders to a worklist or can only query them on-demand as the sample is detected on the machine. That would determine whether the LIS should proactively send orders to the analyzer or wait for them to be requested.

The challenge with that function will be querying the LIMS API when serving both worklist-enabled and on-demand machines. It's possible that the LIS may need to poll the LIMS for all test requests and then store them internally. I'm thinking a SQLite database might be best for that since they provide the application with relational database functionality without needing things like servers. When the LIMS submits orders for tests that are (according to the test code mappings / configuration files) run by a machine that can store orders in a worklist, those orders can be sent to the analyzer right away. Any other orders would continue to wait in the internal database until the LIS is queried by a machine capable of running the ordered test(s).

This approach has advantages and disadvantages. First, the advantages:

  • Since the orders would all be routed by the LIS according to the configured capabilities of each machine, it doesn't require the LIMS to know which machine can run each test.
  • It allows concurrent operation with both worklist-capable and on-demand-only machines.
  • It stores orders internally, potentially reducing delays when the LIS is queried by a machine.
  • If more than one machine is capable of running the ordered test, it would only be sent to one of them. That order would then be marked in the internal database to indicate that it had been relayed to a capable analyzer.

The potential disadvantages that I can see from here:

  • Once the LIS has polled the LIMS for orders on a sample, the LIMS has no way to know the status of each order. If no machine is capable of running a test, it could be waiting indefinitely. (Possible solution: Add an API endpoint for test status queries.)
  • If the LIMS is queried for the latest orders each time an analyzer queries the LIS for a specific sample, the wait will reduce the performance advantage of storing orders internally. (It would still be worthwhile, though.)
  • If more than one connected analyzer is capable of running a test, things could get complicated. We may need to consider how to prioritize in these cases. If no worklist-enabled machine is capable of running the test, then I assume it would be sent to the first capable machine that queries the LIS for that sample's orders. Beyond that, I'm not sure how to approach this problem.

The more I consider it, the more I think the internal database is the best option when using an API. Still, there are some possible issues that will need to be addressed, so let me know if you have any ideas. I'm hoping to get to work on this over the next few weeks, and though I can't make any specific guarantees regarding the time table, I'm excited that someone is interested enough in the program to ask for features like this. I'm looking forward to working out the design details and implementing these new features.

@ubeffiong
Copy link
Author

ubeffiong commented Dec 11, 2022

I think your approach is perfect.

However for the last disadvantage, there are more than two machines that can run the same test.
If the middle way can't handle this, then I may consider to have a separate getorder API from the LMIS for each machine. This means having a separate getorder API for each machine (not a perfect idea). I don't know how possible yet, but I'm thinking it may be an alternative solution to this. So once a particular machine sends an acknowledgment to the LMIS it will not send the order again to that machine.

It is good to mention that the LMIS has an order resend feature. This is to be done by the users in situations the need the order to be resent to the machine. This function when used overrides the check for that particular order and resend it to the LIS middleware.

I would like to ask, for how long will the orders remain in the database?
I'm thinking we shouldn't keep the orders forever on the database. E.g 7 days may be ideal. Since the LMIS has the option to resend order.
We can have this as a configuration file, to decide whether to delete orders or not within a specified time.

Thanks so much for your kind support and active response.

@roy-harmon roy-harmon self-assigned this Dec 31, 2022
@roy-harmon roy-harmon added the enhancement New feature or request label Dec 31, 2022
@roy-harmon
Copy link
Owner

I have a status update for this issue.

As I've worked on implementing the internal SQLite database, it has become clear to me that the best path forward is to focus on the REST API for receiving orders and reporting results. Trying to support a direct connection to an external database is cumbersome when data is stored primarily in the internal database. Providing a REST API is the best way to ensure that the program is flexible, effective, and maintainable.

To that end, I intend to drop support for external database connections within the program itself. This functionality can be moved to a separate executable that can serve as an add-on; it will read from the external database, communicate with the LIS via the new API, and then write any reported results to the external database.

I don't currently have a firm timetable for this change, but I expect to implement it in at least two stages -- direct database connections will be dropped as the API is added, and later, that removed functionality will return as a separate application.

I feel strongly that this is the best way to improve the program. This change should also speed up development as it will remove several obstacles related to direct external database connections (such as the challenge of synchronizing data between internal and external databases, ensuring correct table/field names, and handling SQL errors). It will also open up the program to anyone not using a SQL-based RDBMS. Restoring the functionality as an add-on option will allow for use by those who have a supported database but lack the resources to utilize the REST API.

@hayder-1982
Copy link

HI DEAR
can explane how work?

@roy-harmon
Copy link
Owner

HI DEAR
can explane how work?

Sure.
I'm still working on getting it to run properly as a Windows service, but when the program is running, it will have three main parts. The first is a SQLite database that stores test request and result information. The second piece communicates with connected analyzers and transfers data between them and the program's internal database. The third part is the new API that makes it easy to view or edit the test requests and results in the internal database.
As I mentioned above, there's still an issue with the program when running it as a Windows service, and I've been working on that in my free time (mostly on weekends). If I can't get it to work soon, I may change it to run as an executable instead of a service, since that's much simpler. Then I could continue to explore options for making it run automatically like a Windows service would.

@hayder-1982
Copy link

hayder-1982 commented Oct 22, 2023 via email

@roy-harmon roy-harmon linked a pull request Feb 19, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants