Knoxbits Blog

_FunctionsSkipCleanOutput and FunctionsPreservedDependencies

2022-07-15T07:00:00+00:00

It is common to run into ‘Could not load file or assembly’ errors when developing in-process Azure Functions.

For example, “Message: Could not load file or assembly 'Microsoft.IdentityModel.Protocols, ...” or “Message: Could not load file or assembly 'Microsoft.IdentityModel.Protocols.OpenIdConnect, ...”.

These errors are typically emitted when a function is triggered. Hopefully during development and not in production.

Here’s a full example of the error logged:

[2021-07-18T19:40:20.200Z] Authorization Failed. System.IO.FileNotFoundException caught while validating JWT token.Message: Could not load file or assembly 'System.IdentityModel.Tokens.Jwt, Version=6.7.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'. The system cannot find the file specified.

Both the name and version of the assembly in the error message are important to understanding what’s gone wrong.

The Problem

These errors are typically caused by the in-process Azure Functions runtime when it tries to remove what it believes are redundant assemblies. The runtime includes a bunch of assemblies that it uses. By default it will remove assemblies from the function app that have the same name as assemblies it already has. The runtime is doing that to help improve performance and reduce space.

That behavior doesn’t cause any problem when the assemblies being removed are the same versions as those used by the runtime. But that behavior becomes a problem for the function app developer when version is different that what the runtime is using.

Note that these problems don’t occur in isolated process (out-of-process) Azure Functions because the runtime runs in a separate process than the function app and doesn’t muck with its assemblies.

Fixes

There are two mechanisms that can be used to fix this problem without playing games with the version of the assemblies or NuGet packages that the function app depends on. Both of these require edits to the function app’s .csproj file.

Skip Cleanout (_FunctionsSkipCleanOutput)
Preserve Dependencies (FunctionsPreservedDependencies)

Skip Cleanout (`_FunctionsSkipCleanOutput`)

Add <_FunctionsSkipCleanOutput>true</_FunctionsSkipCleanOutput> to a <PropertyGroup> in the function app’s .csproj file. The runtime will now skip removing redundant assemblies das part of its clean out process.

Example:

<PropertyGroup>
  <_FunctionsSkipCleanOutput>true</_FunctionsSkipCleanOutput>
</PropertyGroup>

Preserve Dependencies (`FunctionsPreservedDependencies`)

Add one or more FunctionsPreservedDependencies elements to an <ItemGroup> in the function app’s .csproj file. That tells the runtime not to remove the specified assembly as part of its clean out process.

You’ll need to add one <FunctionsPreservedDependencies> element for each assembly that the runtime would otherwise be removed by the runtime. That includes assemblies that are indirectly included by your project. So assemblies used by the assemblies your function app uses.

Example:

<ItemGroup>
  <FunctionsPreservedDependencies Include="Microsoft.IdentityModel.JsonWebTokens.dll" />
  <FunctionsPreservedDependencies Include="Microsoft.IdentityModel.Logging.dll" />
  <FunctionsPreservedDependencies Include="Microsoft.IdentityModel.Protocols.dll" />
  <FunctionsPreservedDependencies Include="Microsoft.IdentityModel.Protocols.OpenIdConnect.dll" />
  <FunctionsPreservedDependencies Include="Microsoft.IdentityModel.Tokens.dll" />
  <FunctionsPreservedDependencies Include="System.IdentityModel.Tokens.Jwt.dll" />
</ItemGroup>

Hunting down the assembly names to list in FunctionsPreservedDependencies elements can be time consuming. You’ll need to rebuild and trigger each of functions in the function app to see if you’ve found all of them.

That list will need to be updated as the assemblies or the version of the assemblies used by the function app or the runtime change.

There is a list of assemblies that the in-process Azure Functions runtime uses. That list is maintained at: https://github.com/Azure/azure-functions-host/blob/dev/tools/ExtensionsMetadataGenerator/test/ExtensionsMetadataGeneratorTests/ExistingRuntimeAssemblies.txt

Which fix to Use?

A dev/devops team might decide between skip cleanout and preserving specific assemblies base on the amount of efficiency they desired in the runtime environments for their function app, and on their capacity to handle additional maintenance.

_FunctionsSkipCleanOutput - Use skip cleanout if efficiency-loss due to multiple assemblies being deployed is not a concern. Skipping cleanout doesn’t require additional potential maintenance over time.

FunctionsPreservedDependencies - Use preserve dependencies if efficiency is important, and worth the cost of the updating the FunctionsPreservedDependencies elements as the function app or runtime evolve.

Testing the Fix

No matter which mechanism you chose to fix the problem, you should test the fix by triggering/testing each function in your function app. Or at least those that depend on different assemblies.

Documentation (none)

As of the writing of this post there is no official documentation for _FunctionsSkipCleanOutput or FunctionsPreservedDependencies.

There was some info for FunctionsPreservedDependencies here but the “Configure your build output settings” section, where it was mentioned, has been removed. Sad.

There was an attempt to add some documentation but it was shot down.

References

https://github.com/bryanknox/AzureFunctionsOpenIDConnectAuthSample/issues/27

https://github.com/Azure/azure-functions-vs-build-sdk/issues/397

https://github.com/Azure/azure-functions-host/issues/7061

https://github.com/Azure/azure-functions-host/blob/dev/tools/ExtensionsMetadataGenerator/test/ExtensionsMetadataGeneratorTests/ExistingRuntimeAssemblies.txt

https://github.com/MicrosoftDocs/azure-docs/pull/77460

dotnet user-secrets set CLI Notes

2021-12-09T08:00:00+00:00

In this post I try to provide some helpful information that is not currently included in the official documentation for the ASP.NET Core’s .NET CLI Secret Manager tool’s dotnet user-secrets set command.

Series: ASP.NET Core Secret Manager tool

This post is part of a series about the ASP.NET Core Secret Manager tool that includes:

dotnet user-secrets set
- Applicable Versions
Concepts
Synopsis
Description
Arguments
Options
Examples
- Default project and configuration
- Piped Input
See also
- Other Documentation
Acknowledgments

dotnet user-secrets set

dotnet user-secrets set

The dotnet user-secrets set command is used to set the value of a named secret in a user secrets store.

The dotnet user-secrets set command is part of ASP.NET Core’s .NET CLI Secret Manager tool tool.

Applicable Versions

This article applies to: .NET 6.x SDK and later versions.

The notes in this post are based on my observations and experiments using dotnet user-secrets CLI tool version: 6.0.0-rtm.21526.8+ae1a6cbe225b99c0bf38b7e31bf60cb653b73a52

Concepts

See the Concepts section of dotnet user-secrets CLI Notes.

Synopsis

General synopsis:

Usage: dotnet user-secrets set [arguments] [options]

Arguments:
  [name]   Name of the secret
  [value]  Value of the secret

Options:
  -?|-h|--help                        Show help information
  -v|--verbose                        Show verbose output
  -p|--project <PROJECT>              Path to project. Defaults to searching the current directory.
  -c|--configuration <CONFIGURATION>  The project configuration to use. Defaults to 'Debug'.
  --id <USERSECRETSID>                The user secrets ID to use.

Synopsis for various options:

dotnet user-secrets set [name] [value]
    [-p|--project <PROJECT>]
    [-c|--configuration <CONFIGURATION>]

dotnet user-secrets set [name] [value]
    --id <USERSECRETSID>

dotnet user-secrets set -?|-h|--help

Description

The dotnet user-secrets set command is used to set the value of a named secret in a user secrets store.

The user secrets store to be operated on can be specified in two different ways:

Directly using the Id Option to specify the UserSecretsId.
Indirectly via a Visual Studio project file. The project file in the current directory is used by default, as is the Debug configuration within the project file. See Project Option and Configuration Option below for info on how to override those defaults.

The user secrets store will be created if it does not already exist.

Arguments

[name] [value]

[name] - Name of the secret to set.
[value] - Value to set for the named secret.

Options

Help Option

-?|-h|--help

Show help information for dotnet user-secrets set command.

Configuration Option

-c|--configuration <CONFIGURATION>

Specifies the configuration in the Visual Studio project file that the UserSecretsId to use should be associated with.

Defaults to the configuration named “Debug”.

Ignored if any id option is specified.

The Visual Studio project file’s global UserSecretsId will be used if the specified configuration is not associated with a UserSecretsId in the project file. In that scenario an error will be output if the project file does not have a global UserSecretsId.

Id Option

--id <USERSECRETSID>

Specifies the UserSecretsId of the user secrets store to operate on.

If an id option is specified, then the tool will ignore any project option, configuration option, and it will not look for a Visual Studio project file.

Project Option

-p|--project <PROJECT>

Specifies the path to the Visual Studio project file on the local machine that should be used to determine the user secrets store to operate on.

Defaults to searching the current directory on the local machine for the Visual Studio project.

Ignored if any id option is specified.

Verbose Option

-v|--verbose

Show verbose information in the command’s output.

The verbose information includes:

Project file path.
Secrets file path.
Additional error and diagnostic details for MSBuild related errors.

Examples

Default project and configuration

dotnet user-secrets set ConnectionString "User ID=bob;Password=***"

Sets the ConnectionString secret to the value User ID=bob;Password=***.

Because no project option was specified, the command looks in the Visual Studio project file in the current directory on the local machine.

No configuration option is specified, so the command looks in the project file for the configuration named “Debug” and uses the UserSecretsId associated with that configuration. If no UserSecretsId is associated with the Debug configuration in the project file, then the UserSecretsId in the project file that is NOT associated with any particular configuration is used. That is referred to as the project’s global UserSecretsId. If global UserSecretsId cannot be found then an error like the following is output.

Could not find the global property 'UserSecretsId' in MSBuild project 'D:\Tests\MyProject\MyProject.csproj'. Ensure this property is set in the project or use the '--id' command line option.

Piped Input

The command will accept piped input. Piped input is expected to be in valid JSON format.

type .\my-secrets.json | dotnet user-secrets set

Acknowledgments

Special thanks to Panagiotis Kanavos whose answer on Stack Overflow provided the information I needed to understand the --configuration option behaves for most of the dotnet user-secrets commands.

dotnet user-secrets init CLI Notes

2021-12-08T08:00:00+00:00

Series: ASP.NET Core Secret Manager tool

This post is part of a series about the ASP.NET Core Secret Manager tool that includes:

dotnet user-secrets init
- Applicable Versions
Concepts
Synopsis
Description
- Configuration-specific User Secrets
Options
Examples
See also
- Other Documentation
Acknowledgments

dotnet user-secrets init

The dotnet user-secrets init command is used to initialize or update a Visual Studio project file so that the project can use secrets stored in a user secrets store.

The command is part of ASP.NET Core’s .NET CLI Secret Manager tool tool.

Applicable Versions

This article applies to: .NET 6.x SDK and later versions.

The notes in this post are based on my observations and experiments using dotnet user-secrets CLI tool version: 6.0.0-rtm.21526.8+ae1a6cbe225b99c0bf38b7e31bf60cb653b73a52

Concepts

See the Concepts section of dotnet user-secrets CLI Notes.

Synopsis

General synopsis:

Usage: dotnet user-secrets init [options]

Options:
  -?|-h|--help                        Show help information.
  -v|--verbose                        Ignored.
  -p|--project <PROJECT>              Path to project. Defaults to searching the current directory.
  -c|--configuration <CONFIGURATION>  Ignored.
  --id <USERSECRETSID>                The user secrets ID to use.

Synopsis for various options:

dotnet user-secrets init
    [--id <USERSECRETSID>]
    [-p|--project <PROJECT>]

dotnet user-secrets init -?|-h|--help

Description

The dotnet user-secrets init command is used to initialize or update a Visual Studio project file so that the project can use secrets stored in a user secrets store.

The init command does NOT create a user secrets store.

The command adds or updates the Visual Studio project file’s global UserSecretsId.

The global UserSecretsId is the <UserSecretsId> element in the first <PropertyGroup> element in the Visual Studio project file that does not have a Condition attribute.

If no <PropertyGroup> element exists in the project file, or if all <PropertyGroup> elements have Condition attributes, then a <PropertyGroup> element (with no Condition attribute) is added to the project file, and the <UserSecretsId> element is added to it.

The <UserSecretsId> element’s inner text is initialized to new GUID by default, unless a UserSecretsId is specified by the --id <USERSECRETSID> option.

If the Project Option is not specified then the tool searches the current directory for the Visual Studio project file.

Configuration-specific User Secrets

The global UserSecretsId added or updated in the project by the tool is independent of any configuration associated with the project. It will be used by any configuration that does not have an associated UserSecretsId setup in the project file.

If you want to use configuration-specific user secrets stores, then you’ll need to manually add <UserSecretsId> elements to the <PropertyGroup> elements that have Condition attributes the associate the <PropertyGroup> with the configuration.

See the Configuration Option section below for more info.

Options

Help Option

-?|-h|--help

Show help information for dotnet user-secrets init command.

Configuration Option

-c|--configuration <CONFIGURATION>

Ignored. Currently (in version 6.0.0-rtm.21526.8+ae1a6cbe225b99c0bf38b7e31bf60cb653b73a52) the -c|--configuration <CONFIGURATION> options seems to be ignored and have no effect on the dotnet user-secrets init command. Or, it could be that I haven’t figured out how to properly use the --configuration option with the dotnet user-secrets init command.

Other dotnet user-secrets commands do support the configuration option.

Manually adding Configuration-specific UserSecretsIds

If you want to use configuration-specific user secrets stores, then you’ll need to manually edit the Visual Studio project file and add <UserSecretsId> elements to the <PropertyGroup> elements with Condition attributes that associate the <PropertyGroup> with the configuration.

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net5.0</TargetFramework>
    <!-- Global UserSecretsId -->
    <UserSecretsId>myGlobal-UserSecretsId</UserSecretsId>
  </PropertyGroup>

  <PropertyGroup Condition="'$(Configuration)'=='TestConfig1'">
    <!-- TestConfig1 configuration's  UserSecretsId -->
    <UserSecretsId>myTestConfig1-UserSecretsId</UserSecretsId>
  </PropertyGroup> 

  <PropertyGroup Condition="'$(Configuration)'=='TestConfig2'">
    <!-- TestConfig2 configuration's  UserSecretsId -->
    <UserSecretsId>myNewUserSecretId</UserSecretsId>
  </PropertyGroup>

My configuration option wish

I would like to see the --configuration options be supported by the dotnet user-secrets init command so that it could be used to initialize or update the global UserSecretsId or a configuration-specific UserSecretsId.

The following is a sample Visual Studio project file that has a global UserSecretsId, plus configuration-specific user secrets IDs for configurations named TestConfig1 and TestConfig2.

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net5.0</TargetFramework>
    <!-- Global UserSecretsId -->
    <UserSecretsId>myGlobal-UserSecretsId</UserSecretsId>
  </PropertyGroup>

  <PropertyGroup Condition="'$(Configuration)'=='TestConfig1'">
    <!-- TestConfig1 configuration's  UserSecretsId -->
    <UserSecretsId>myTestConfig1-UserSecretsId</UserSecretsId>
  </PropertyGroup> 

  <PropertyGroup Condition="'$(Configuration)'=='TestConfig2'">
    <!-- TestConfig2 configuration's  UserSecretsId -->
    <UserSecretsId>myTestConfig2-UserSecretsId</UserSecretsId>
  </PropertyGroup>

Example - Update UserSecretsId for Specified Configuration

The following command would update the sample project above by setting the the user secrets ID in the <UserSecretsId> element within the <PropertyGroup> element where Configuration is TestConfig2.

dotnet user-secrets init -c TestConfig1 -id myNewUserSecretId

Id Option

--id \<USERSECRETSID>

Specifies the UserSecretsId to be set in the Visual Studio project file.

If the Id option is not specified then the tool defaults to using a new GUID as the UserSecretsId.

The specified UserSecretsId is set as the inner text of the <UserSecretsId> element added or updated in the Visual Studio project file.

The UserSecretsId to used can be simple text, it does not need to be a GUID.

The user secrets store associated with the specified UserSecretsId does not need to already exist.

The dotnet user-secrets init command does NOT create the user secrets store associated with the UserSecretsId.

Currently (version 6.0.0-rtm.21526.8+ae1a6cbe225b99c0bf38b7e31bf60cb653b73a52) the UserSecretsId is used in the file path of the user secrets store’s secret.json file.

Warning: The UserSecretsId should contain only valid file path characters for the operating systems of the machines where the Visual Studio project file will be used.

The dotnet user-secrets init command does not enforce that requirement and will succeed in setting a UserSecretsId, but other dotnet user-secrets commands will produce errors like the following example when they encounter such UserSecretsIds.
Command failed : Invalid character ':' found in the `UserSecretsId` at index '3'.
The limitation is required because in the current Secret Manager tool (version 6.0.0-rtm.21526.8+ae1a6cbe225b99c0bf38b7e31bf60cb653b73a52) implementation the UserSecretsId is used to form the file path to the user secrets store’s secret.json file.

Project Option

-p|--project <PROJECT>

Path to the Visual Studio project file where the UserSecretsId will be added or updated.

If the project option is not specified then the tool defaults to searching the current directory for the Visual Studio project file.

Verbose Option

-v|--verbose

Ignored. As of version 6.0.0-rtm.21526.8+ae1a6cbe225b99c0bf38b7e31bf60cb653b73a52, the -v|--verbose options seems to be ignored and have no effect on the output of the dotnet user-secrets init command.

Examples

Initialize using defaults

dotnet user-secrets init

Generates a new GUID will be used as the UserSecretsId.

<PropertyGroup>
  <TargetFramework>net6.0</TargetFramework>
  <UserSecretsId>03c27eab-1b9f-40b3-b8a0-2c68b57fe526</UserSecretsId>
</PropertyGroup>

If the Visual Studio project file already has a UserSecretsId, it will not be updated.

Specify Visual Studio project to initialize

Use the -p|--project <PROJECT> option to specify the path to the Visual Studio project files that the UserSecretsId should be added to.

dotnet user-secrets init -p D:/repos/my-repos/sample/MyProject/MyProject.cspoj`

Specify custom user secrets ID

Use the --id <USERSECRETSID> option to specify the UserSecretsId to add or update in the Visual Studio project file.

dotnet user-secrets init `--id local-dev-test1`

The following Visual Studio project snippet shows an example of the <UserSecretsId> element that is added or updated. Where local-dev-test1 is the UserSecretsId that was specified.

<PropertyGroup>
  <TargetFramework>net6.0</TargetFramework>
  <UserSecretsId>local-dev-test1</UserSecretsId>
</PropertyGroup>

Visual Studio projects with PropertyGroup Condition attributes

dotnet user-secrets init --id my-user-secret-id

In the Visual Studio project file, a <UserSecretsId> element is added or updated in the first <PropertyGroup> element that does not have a Condition attribute. The <UserSecretsId> element’s inner text is set to my-user-secret-id.

<PropertyGroup Condition="'$(Configuration)'=='Release'">
    <!-- First PropertyGroup with a Condition. -->
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)'=='Debug'">
    <!-- Second PropertyGroup with a Condition. -->
</PropertyGroup>
<PropertyGroup>
  <!-- First PropertyGroup without a Condition. -->
  <UserSecretsId>my-user-secret-id</UserSecretsId>
</PropertyGroup>
<PropertyGroup>
  <!-- Second PropertyGroup without a Condition. -->
</PropertyGroup>

Acknowledgments

Special thanks to Panagiotis Kanavos whose answer on Stack Overflow provided the information I needed to understand the --configuration option behaves for most of the dotnet user-secrets commands.

dotnet user-secrets CLI Notes

2021-12-07T08:00:00+00:00

In this post I try to provide some helpful information that is not currently included in the official documentation for the ASP.NET Core’s .NET CLI dotnet user-secrets tool (a.k.a. Secret Manager tool).

Series: ASP.NET Core Secret Manager tool

This post is part of a series about the ASP.NET Core Secret Manager tool that includes:

dotnet user-secrets
- Applicable Versions
- Secret Manager vs Secrets Manager
Concepts
Synopsis
Description
Options
Commands
- Commands for managing user secrets IDs in a Visual Studio project
- Commands for managing secrets in a user secrets store
Examples
- Show Help
- Show Version
Implementation Details
- secret.json user secrets store
See also
- Other Documentation
- Source Code for dotnet-user-secrets Tool
Acknowledgments

dotnet user-secrets

dotnet user-secrets is a command line tool for managing the set of secrets in a user secrets store, and managing the user secrets store used by a Visual Studio project.

The dotnet user-secrets tool (a.k.a. Secret Manager tool) is a .NET CLI tool that is part of ASP.NET Core.

Applicable Versions

This article applies to: .NET 6.x SDK and later versions.

The notes in this post are based on my observations and experiments using dotnet user-secrets CLI tool version: 6.0.0-rtm.21526.8+ae1a6cbe225b99c0bf38b7e31bf60cb653b73a52

Secret Manager vs Secrets Manager

The Microsoft docs call it “Secret Manager”, while dotnet user-secrets --help calls it “Secrets Manager”.

Concepts

Secrets and user secrets stores

A secret has a name and a value.
Sets of secrets are stored in a user secrets store.
- The implementation and location of a user secrets store are hidden behind the abstractions provided by the Secret Manager tool. That allows the Secret Manager to be used by developer and code without exposing the implementation details. It also allows the tool to evolve to support user secrets stores in different locations or with different implementations.
  - See the Implementation Details section below.
  - In these notes I use the term “user secrets store” instead of “secrets file” to honor the intended abstractions.
Each individual secret has a unique name that is used to identify it within the user secrets store.

User secrets stores and user secrets IDs

A user secrets ID is used to identify a user secrets store.
- Developers can specify a UserSecretsId to the dotnet user-secrets tool to manage secrets in the user secrets store associated with that UserSecretsId.

Project files, configurations and user secrets IDs

The user secrets store to be operated on by the dotnet user-secrets tool can be specified in two different ways:

Directly using the Id option to specify the UserSecretsId.
Indirectly via a Visual Studio project file.

A UserSecretsId can be added to a Visual Studio project file to associate the project with a specific user secrets store.
When a Visual Studio project file has a UserSecretsId the project’s code can use secrets in the associated user secrets store via ASP.NET Core’s Secret Manager.
The dotnet user-secrets tool can read the UserSecretsId from a specified Visual Studio project file, or the tool can search for a project file from which to read the UserSecretsId.
- Developers can use the dotnet user-secrets tool to manage secrets in the user secrets store associated with the project, without having to explicitly specify the UserSecretsId.
A Visual Studio project file can have multiple configurations and each build of the project uses exactly one of the configurations.
A configuration in a Visual Studio project can be associated with a UserSecretsId. That allows the build of the project for a particular configuration to use secrets from the user secrets store associated with that configuration.
- Developers can specify a configuration to the the dotnet user-secrets tool and it will use it to search the the Visual Studio project file for the UserSecretsId that should be used.
The use of configurations is optional.
- Developers can setup a Visual Studio project file so that the same UserSecretsId is used regardless of the number of configurations that have been setup for the project.
Configurations are independent of user secrets stores.
- Each configuration can be setup in the project with a different UserSecretsId, so that each configuration uses a different store. Or multiple configurations in the same project can use the same UserSecretsId, so that they use the same user secrets store.
The combination of project and configuration can be used by the dotnet user-secrets tool to determine the UserSecretsId to use for accessing the associated user secrets store.
- Projects and configurations are used as shortcuts that can be a convenient way for developers to work with secrets in user secrets stores. Once the project file is setup to associate user secrets IDs with its configurations then the developers can use the dotnet user-secrets tool and indicate a configuration. That allows the developer manage the secrets for a particular project configuration without having to remember the specific UserSecretsId.

Synopsis

General synopsis:

Usage: dotnet user-secrets [command] [options]

Options:
  -?|-h|--help                        Show help information
  --version                           Show version information
  -v|--verbose                        Show verbose output
  -p|--project <PROJECT>              Path to project. Defaults to searching the current directory.
  -c|--configuration <CONFIGURATION>  The project configuration to use. Defaults to 'Debug'.
  --id <USERSECRETSID>                The user secrets ID to use.

Commands:
  clear   Deletes all the secrets in a user secrets store.
  init    Initialize or update a Visual Studio projectto use a specified user secrets store.
  list    Lists secrets in a user secrets store. 
  remove  Removes the specified secret from a user secrets store.
  set     Sets a secret to a specified value in a user secrets store.

Synopsis for usage without a command:

dotnet user-secrets -?|-h|--help|-v|-verbose

dotnet user-secrets --version

Description

The dotnet user-secrets tool can be used for:

Managing secrets in a user secrets store.
- See Commands for managing secrets in a user secrets store below.
Initializing or updating a Visual Studio project file so that the project can use secrets stored a specified user secrets store.
- See dotnet user-secrets init CLI Notes

Options

The options described below are only those for dotnet user-secrets when no additional command is specified. For details about options for specific commands, follow the links in the Commands section below.

-?|-h|--help

Show dotnet user-secrets tool help information.
--version

Show dotnet user-secrets tool version information.
-v|--verbose

Show verbose output. When no command is specified the output is the same as for --help.

Commands

For details about specific commands and their options follow the links below.

Commands for managing user secrets IDs in a Visual Studio project

init - Initialize or update a Visual Studio project file so that the project can use secrets stored a specified user secrets store.
- dotnet user-secrets init [options]
- See dotnet user-secrets init CLI Notes

Commands for managing secrets in a user secrets store

clear - Deletes all the secrets in a user secrets store.
- dotnet user-secrets clear [options]
list - Lists secrets in a user secrets store.
- dotnet user-secrets list [options]`
remove - Removes the specified secret from a user secrets store.
- dotnet user-secrets remove [arguments] [options]
set - Sets a secret to a specified value in a user secrets store.
- dotnet user-secrets set [arguments] [options]
- See dotnet user-secrets set CLI Notes

I haven’t written posts for the clear, list, and remove commands, but those should be fairly easy to figure out from the information in dotnet user-secrets set CLI Notes.

Examples

The following are examples of the dotnet user-secrets tool without a specified command.

Show Help

Show dotnet user-secrets tool help.

Command Format:

dotnet user-secrets -?|-h|--help|-v|-verbose

All of the options above the produce same output.

Example:

dotnet user-secrets --help

Output:

User Secrets Manager 6.0.0-rtm.21526.8+ae1a6cbe225b99c0bf38b7e31bf60cb653b73a52

Usage: dotnet user-secrets [options] [command]

Options:
  -?|-h|--help                        Show help information
  --version                           Show version information
  -v|--verbose                        Show verbose output
  -p|--project <PROJECT>              Path to project. Defaults to searching the current directory.
  -c|--configuration <CONFIGURATION>  The project configuration to use. Defaults to 'Debug'.
  --id                                The user secrets ID to use.

Commands:
  clear   Deletes all the application secrets
  init    Set a user secrets ID to enable secret storage
  list    Lists all the application secrets
  remove  Removes the specified user secret
  set     Sets the user secret to the specified value

Use "dotnet user-secrets [command] --help" for more information about a command.

Show Version

Show dotnet user-secrets tool version information.

Example:

dotnet user-secrets --version

Output:

User Secrets Manager
6.0.0-rtm.21526.8+ae1a6cbe225b99c0bf38b7e31bf60cb653b73a52

Implementation Details

secret.json user secrets store

The implementation and location of a user secrets store are hidden behind the abstractions provided by the Secret Manager tool. That allows the Secret Manager to be used by developer and code without exposing the implementation details. It also allows the tool to evolve to support user secrets stores in different locations or with different implementations.

Developers are warned not to write code that depends on the location, storage or implementation details of user secrets stores as those things could changes in the future.

Currently (version 6.0.0-rtm.21526.8+ae1a6cbe225b99c0bf38b7e31bf60cb653b73a52) user secrets stores are implemented as JSON files named secret.json that are stored in the local machine’s user profile folder.

File system path:

Linux/macOS: ~/.microsoft/usersecrets/<user_secrets_id>/secrets.json

Windows: %APPDATA%\Microsoft\UserSecrets\<user_secrets_id>\secrets.json

Where <user_secrets_id> is the user secrets ID that is used to uniquely identify the user secrets store on the local machine.

Acknowledgments

Special thanks to Panagiotis Kanavos whose answer on Stack Overflow provided the information I needed to understand the --configuration option behaves for most of the dotnet user-secrets commands.

Duplicate Detection for Service Bus output binding in Azure Functions

2020-04-01T00:00:00+00:00

Here’s a way to get the Azure Service Bus output binding for your Azure Function working after you’ve enabled Duplicate Detection (or other features) on the Service Bus queue or topic.

If you haven’t had duplicate detection turned on or haven’t been using some of the other Azure Service Bus features, you might set up the output binding in your Azure Functions so you can send a custom class object to a Service Bus queue or topic. Something like the following.

[FunctionName("MyFunction1")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "get", Route = null)] HttpRequest req,
[ServiceBus("%MyQueueName%",
Connection = "MyServiceBusConnection")] IAsyncCollector<MyItem> myOutputQueue)
{
    var myItem = new MyItem();

    // Do stuff.
    // . . .

    // Add the item to the queue.
    await myOutputQueue.AddAsync(myItem);

    // Do more stuff.
    // . . .

    return new OkResult();
}

Notice that the Service Bus out parameter specifies the custom class we want to send to the to the queue or topic. In the example the custom class is MyItem.

IAsyncCollector<MyItem>

That will work great as long as you don’t have Duplicate Detection (or some of the other features) enabled for the queue or topic.

See Azure Service Bus Docs: Duplicate detection and Messages, payloads, and serialization for more information about Service Bus Duplicate Detection.

Throws When Service Bus has Duplicate Detection Enabled

If you enable Duplicate Detection on the queue or topic, your function will throw an System.InvalidOperationException that has a message that starts with:

Message to a partitioned entity with duplicate detection enabled must have either PartitionKey or MessageId set . . .

A Solution to Support Service Bus Duplicate Detection

One way to make it work when duplicate detection is enabled is to output a Micosoft.Azure.ServiceBus.Message that contains your custom class, and then set the MessageId on the Message object to be added to the queue or topic.

This approach will work whether or not duplicate detection is enabled on the queue or topic. You can also use the approach to take advantage of other Azure Service Bus features that require the settings of properties on the Micosoft.Azure.ServiceBus.Message objects.

[FunctionName("MyFunction2")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "get", Route = null)] HttpRequest req,
[ServiceBus("%MyQueueName%",
Connection = "MyServiceBusConnection")] IAsyncCollector<Message> myOutputQueue)
{
    var myItem = new MyItem();

    // Do stuff.
    // . . . 

    // Add the item to the queue, wrapped in a Message.
    string json = JsonConvert.SerializeObject(myItem);
    Message outputMessage = new Message(Encoding.UTF8.GetBytes(json))
    {
        ContentType = "application/json",

        // Set the MessageId for use in duplicate detection.

        MessageId = $"{myItem.TypeId}-{myItem.ItemId}" 
    };

    await myOutputQueue.AddAsync(outputMessage);

    // Do more stuff.
    // . . .

    return new OkResult();
}

Notice that we changed the Service Bus out parameter class type to Micosoft.Azure.ServiceBus.Message.

    IAsyncCollector<Message>

And we wrapped our custom class object (MyItem in the example) in a Message, and set its MessageId property.

    string json = JsonConvert.SerializeObject(myItem);
    Message outputMessage = new Message(Encoding.UTF8.GetBytes(json))
    {
        ContentType = "application/json",

        // Set the MessageId for use in duplicate detection.

        MessageId = $"{myItem.TypeId}-{myItem.ItemId}" 
    };

    await myOutputQueue.AddAsync(outputMessage); 

Under the Hood of the Microsoft.Azure.WebJobs.Extensions.ServiceBus

The source code in GitHub for Microsoft.Azure.WebJobs.Extensions.ServiceBus shows how it handles the difference between a custom class and Microsoft.Azure.ServiceBus.Message.

See MessageConverterFactory.cs at: https://github.com/Azure/azure-functions-servicebus-extension/blob/master/src/Microsoft.Azure.WebJobs.Extensions.ServiceBus/Bindings/MessageConverterFactory.cs

And UserTypeToBrokeredMessageConverter.cs at: https://github.com/Azure/azure-functions-servicebus-extension/blob/master/src/Microsoft.Azure.WebJobs.Extensions.ServiceBus/Bindings/UserTypeToBrokeredMessageConverter.cs

When a custom class is detected by the MessageConverterFactory it calls the UserTypeToBrokeredMessageConverter to create a Micosoft.Azure.ServiceBus.Message to wrap it. So we can see that under the hood, everything put in a Azure Service Bus queue or topic is a Message.

Knoxbits Blog

_FunctionsSkipCleanOutput and FunctionsPreservedDependencies

The Problem

Fixes

Skip Cleanout (_FunctionsSkipCleanOutput)

Preserve Dependencies (FunctionsPreservedDependencies)

Which fix to Use?

Testing the Fix

Documentation (none)

References

dotnet user-secrets set CLI Notes

Series: ASP.NET Core Secret Manager tool

Table of Contents

dotnet user-secrets set

Applicable Versions

Concepts

Synopsis

Description

Arguments

Options

Help Option

Configuration Option

Id Option

Project Option

Verbose Option

Examples

Default project and configuration

Piped Input

See also

Other Documentation

Acknowledgments

dotnet user-secrets init CLI Notes

Series: ASP.NET Core Secret Manager tool

Table of Contents

dotnet user-secrets init

Applicable Versions

Concepts

Synopsis

Description

Configuration-specific User Secrets

Options

Help Option

Configuration Option

Manually adding Configuration-specific UserSecretsIds

My configuration option wish

Example - Update UserSecretsId for Specified Configuration

Id Option

Project Option

Verbose Option

Examples

Initialize using defaults

Specify Visual Studio project to initialize

Specify custom user secrets ID

Visual Studio projects with PropertyGroup Condition attributes

See also

Other Documentation

Acknowledgments

dotnet user-secrets CLI Notes

Series: ASP.NET Core Secret Manager tool

Table of Contents

dotnet user-secrets

Applicable Versions

Secret Manager vs Secrets Manager

Concepts

Secrets and user secrets stores

User secrets stores and user secrets IDs

Project files, configurations and user secrets IDs

Synopsis

Description

Options

Commands

Commands for managing user secrets IDs in a Visual Studio project

Commands for managing secrets in a user secrets store

Examples

Show Help

Show Version

Implementation Details

secret.json user secrets store

See also

Other Documentation

Skip Cleanout (`_FunctionsSkipCleanOutput`)

Preserve Dependencies (`FunctionsPreservedDependencies`)