After all fiddling it stands out: there is no real solution for this issue; at least none that is not very hackish or without changing the whole setup/workflow.
For the sake of completeness I leave my "answers" as they are (see everything below "LOG") and add some background information.
One more idea - very hackish
You can search for all scripts in the string that will be appended. Then search for all "$( function(){...} )" occurrences with regex then insert a function like "$( function(){...;executionHandler()} )" that will count down until all these constructs are resolved and then set an outer promise as resovled. But this would be quite hackish as I said at the beginning and also might be quite error prone.
LOG
Version 1, 2, 2.1 and 3 are all tested with jQuery versions 1.12.4, 2.2.4 and 3.2.1 and should work fine with all versions from jQuery 1.8 and above.
$.get('...', function(){
    // make shure that $Deferred is in scope of all involved functions
    window.$Deferred = $.Deferred(); // get jQuery deferred object
    $('#scripts').append(
        "hi<script>$(function(){$('#log').append('<p>3</p>');});<\/script>" + // append your common stuff
        "<script>$(function(){$Deferred.resolve();})<\/script>" // add this line to resolve window.$Deferred
    );
    $Deferred.always(function() {
        // execute code after $Deferred was resolved or rejected
        $('#log').append('<p>1</p>');
        $('#log').append('<p>2</p>');
    });
});
was kicked
// $.get returns a jqXHR object which is a promise-compatible object
$.when($.get('...', function() {
    $('#scripts').append(
        "hi<script>$(function(){$('#log').append('<p>3</p>');});<\/script>"
    )
})).always(function( data, textStatus, jqXHR|errorThrown ) {
    // execute code when $.get is done or fails
    $('#log').append('<p>1</p>');
    $('#log').append('<p>2</p>');
});
Or alternatively
$.when($.get('...', function() {
    $('#scripts').append(
        "hi<script>$(function(){$('#log').append('<p>3</p>');});<\/script>"
    )
})).then(
    function( data, textStatus, jqXHR ) {
        // success callback
    },
    function( jqXHR, textStatus, errorThrown ) {
        // fail callback
    }
);
Details
- jQuery.when( deferreds )- 
- Provides a way to execute callback functions based on zero or more Thenable objects, usually Deferred objects that represent asynchronous events.
 [...]
 For example, the jqXHR object returned by jQuery.ajax() is a Promise-compatible object and can be used [...]
 
 
- jQuery.get( url [, data ] [, success ] [, dataType ] )
 - 
- [...] is a shorthand Ajax function [...] 
 
- deferred.always( alwaysCallbacks [, alwaysCallbacks ] )
 - 
- Add handlers to be called when the Deferred object is either resolved or rejected. 
 
- deferred.then( doneFilter [, failFilter ] [, progressFilter ] )- 
- Add handlers to be called when the Deferred object is resolved, rejected, or still in progress. 
 
References