Swift Package Manager - How to load a Swift Dynamic Library without the source on Linux (.so file)

Question

I am currently developing a Swift app on Linux, and I am unable to load a Swift Package using the .so file instead of linking directly to the source (locally/github). The library has pure Swift code.

Package.swift file from the library,

import PackageDescription

let package = Package(
        name: "LibraryName",
        products: [
            .library(
                    name: "LibraryName",
                    targets: ["LibraryName"]),
        ],
        dependencies: [

        ],
        targets: [
            // Targets are the basic building blocks of a package. A target can define a module or a test suite.
            // Targets can depend on other targets in this package, and on products in packages this package depends on.
            .target(
                    name: "AppName",
                    dependencies: [

                    ]),
        ]
)

When I run swift build command, I get the 4 files (LibraryName.so, LibraryName.swiftdoc, LibraryName.swiftsourceinfo, LibraryName.swiftmodule).

I have tried linking the .so/.swiftmodule files directly using .binaryTarget, but that seems to only accept .xcframework files. I have found some other answers suggesting adding static (.a) libraries directly in XCode. I am developing on Ubuntu using CLion, so I am not able to do that either.

I know that Swift ABI has it's issues when the libraries are compiled with different compiler versions, but for my use case, it's a non-issue.

In C++, I used to be able to share the .so files along with the header files. Is something similar possible using Swift?