Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add bpf2go support and wrappers for ebpf.Variable generation #1543

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

smagnani96
Copy link
Contributor

@smagnani96 smagnani96 commented Aug 8, 2024

This commit introduces support and wrappers around ebpf.Variable.
An ebpf.Variable does not define per-se wrappers method to automatically
infer the underlying data type and handle typed data in Go. In fact, it
relies on parameters passed to the Variable.Get and Variable.Set APIs.
This commit allows bpf2go to emit VariableSpec and Variable in the
generated code. In addition, a new set of wrapper methods are created
to provide support for typed method around variables. To do so, this
commit enables emitting Dataspec, which so far was not emitted. However,
it modifies the current implementation so that for each variable in Dataspec
the needed support is provided.
Supposing to have an ebpf program that defines the following two global variables:

__u64 pkt_count			= 0;
const __u32 ro_variable	= 1;
char var_msg[]			= "Hello World!";

The steps performed during the Go code generation are as follows:

  1. An alias with the variable name is generated in the Go file:
type PktCountType 	= uint64
type RoVariable 	= uint32
type VarMsgType 	= [16]int8
  1. A new type definition (non-alias, need new methods) for the variables is performed:
type PktCount	ebpf.Variable
type RoVariable	ebpf.Variable
type VarMsg		ebpf.Variable
  1. The .Get methods for all types are generated:
func (v *PktCount) Get() (PktCountType, error) {
	var ret PktCountType
	return ret, (*ebpf.Variable)(v).Get(&ret)
}

func (v *RoVariable) Get() (RoVariableType, error) {
	var ret RoVariableType
	return ret, (*ebpf.Variable)(v).Get(&ret)
}

func (v *VarMsg) Get() (VarMsgType, error) {
	var ret VarMsgType
	return ret, (*ebpf.Variable)(v).Get(&ret)
}
  1. The .Set methods for the non read-only variables are generated:
func (v *PktCount) Set(val PktCountType) error {
	return (*ebpf.Variable)(v).Set(val)
}

func (v *VarMsg) set(val VarMsgType) error {
	return (*ebpf.Variable)(v).Set(ret)
}
  1. In case the type alias is supported by the atomic package, and then
    also supported by ebpf.Variable (int32, uint32, int64, uint64), then
    an additional AtomicRef method is created for the non read-only
    variables to get an reference to the underlying data.
func (v *PktCount) AtomicRef() (*ebpf.Uint64, error) {
	ret, _ := (*ebpf.Variable)(v).AtomicUint64()
	return ret
}

From the user perspective, ignoring the error catching, the following
operations can be performed:

...
objs := bpfObjects{}
err := loadBpfObjects(&objs, nil)
...

// ref kept and can be used multiple times
aPktCount := objs.PktCount.AtomicRef()
vPktCount := aPktCount.Load()
aPktCount.Store(rand.Uint64())
...

vRoVariable, _ := objs.RoVariable.Get()
...

varMsg, _ := objs.VarMsg.Get()
varMsg[0] = 'B'
err := objs.VarMsg.Set(varMsg)

Suggestions/Comments are very welcome. For general discussion, I've opened the issue (:arrow_down:).

This PR relies and is rebased on https://github.com/ti-mo/ebpf/tree/tb/ebpf-variables, not yet merged but planned. Additionally, I'd need the support introduced in my 1st commit 59808a1.

The only new commit that should be referred in this PR is the latest f6a94c5.

My old Variable API proposal prior to https://github.com/ti-mo/ebpf/tree/tb/ebpf-variables are kept here.

Fixes: #1542

@github-actions github-actions bot added the breaking-change Changes exported API label Aug 8, 2024
@github-actions github-actions bot removed the breaking-change Changes exported API label Aug 8, 2024
@smagnani96 smagnani96 force-pushed the pr/relocated_vars_userspace branch 2 times, most recently from fa76e79 to 71e3029 Compare August 14, 2024 10:18
@smagnani96 smagnani96 force-pushed the pr/relocated_vars_userspace branch 3 times, most recently from 2cf61df to 9cf6f9f Compare September 5, 2024 12:51
@smagnani96 smagnani96 changed the title support for interacting with global/static vars add bpf2go support and wrappers for ebpf.Variable generation Sep 5, 2024
@smagnani96
Copy link
Contributor Author

smagnani96 commented Sep 5, 2024

This PR has been rebased on https://github.com/ti-mo/ebpf/tree/tb/ebpf-variables.
After discussing, @ti-mo proceeds with the Map.Memory() api and the definition of ebpf.Variable, while me (this PR) should provide the needed support for bpf2go.

  1. 59808a1 is required but should be introduced in ti-mo updates (open for discussion).
  2. f6a94c5 is the commit under review.

…ndler

Signed-off-by: Timo Beckers <timo@isovalent.com>
Signed-off-by: Timo Beckers <timo@isovalent.com>
…iables

Signed-off-by: Timo Beckers <timo@isovalent.com>
Signed-off-by: Timo Beckers <timo@isovalent.com>
Signed-off-by: Timo Beckers <timo@isovalent.com>
Signed-off-by: Timo Beckers <timo@isovalent.com>
Signed-off-by: Simone Magnani <simone.magnani@isovalent.com>
@smagnani96 smagnani96 force-pushed the pr/relocated_vars_userspace branch 2 times, most recently from 8486b44 to f6a94c5 Compare September 6, 2024 21:31
This commit introduces support and wrappers around ebpf.Variable.
An ebpf.Variable does not define per-se wrappers method to automatically
infer the underlying data type and handle typed data in Go. In fact, it
relies on parameters passed to the `Variable.Get` and `Variable.Set` APIs.
This commit allows `bpf2go` to emit `VariableSpec` and `Variable` in the
generated code. In addition, a new set of wrapper methods are created
to provide support for typed method around variables. To do so, this
commit enables emitting `Dataspec`, which so far was not emitted. However,
it modifies the current implementation so that for each variable in `Dataspec`
the needed support is provided.
Supposing to have an ebpf program that defines the following two global variables:

```c
__u64 pkt_count			= 0;
const __u32 ro_variable	= 1;
char var_msg[]			= "Hello World!";
```

The steps performed during the Go code generation are as follows:

1. An alias with the variable name is generated in the Go file:

```go
type PktCountType 	= uint64
type RoVariable 	= uint32
type VarMsgType 	= [16]int8
```

2. A new type definition (non-alias, need new methods) for the variables is performed:

```go
type PktCount	ebpf.Variable
type RoVariable	ebpf.Variable
type VarMsg		ebpf.Variable
```

3. The `.Get` methods for all types are generated:

```go
func (v *PktCount) Get() (PktCountType, error) {
	var ret PktCountType
	return ret, (*ebpf.Variable)(v).Get(&ret)
}

func (v *RoVariable) Get() (RoVariableType, error) {
	var ret RoVariableType
	return ret, (*ebpf.Variable)(v).Get(&ret)
}

func (v *VarMsg) Get() (VarMsgType, error) {
	var ret VarMsgType
	return ret, (*ebpf.Variable)(v).Get(&ret)
}
```

4. The `.Set` methods for the non read-only variables are generated:

```go
func (v *PktCount) Set(val PktCountType) error {
	return (*ebpf.Variable)(v).Set(val)
}

func (v *VarMsg) set(val VarMsgType) error {
	return (*ebpf.Variable)(v).Set(ret)
}
```

4. In case the type alias is supported by the atomic package, and then
    also supported by `ebpf.Variable` (int32, uint32, int64, uint64), then
    an additional `AtomicRef` method is created for the non read-only
	variables to get an reference to the underlying data.

```go
func (v *PktCount) AtomicRef() (*ebpf.Uint64, error) {
	ret, _ := (*ebpf.Variable)(v).AtomicUint64()
	return ret
}
```

From the user perspective, ignoring the error catching, the following
operations can be performed:

```go
...
objs := bpfObjects{}
err := loadBpfObjects(&objs, nil)
...

// ref kept and can be used multiple times
aPktCount := objs.PktCount.AtomicRef()
err := aPktCount.Load()
aPktCount.Store(rand.Uint64())
...

vRoVariable, _ := objs.RoVariable.Get()
...

varMsg, _ := objs.VarMsg.Get()
varMsg[0] = 'B'
err := objs.VarMsg.Set(varMsg)
```

Signed-off-by: Simone Magnani <simone.magnani@isovalent.com>
@mejedi
Copy link
Contributor

mejedi commented Sep 26, 2024

It probably makes sense to land basic bpf2go variable support first:

  • VariableSpecs included in generated xxxSpecs; do we need to segregate them similar to today's xxxProgramSpecs/xxxMapSpecs?
  • Variables included in generated xxxObjects.

I see huge value in a type-safe get/set API you are proposing. It would be nice to have not only Variables covered, but probably VariableSpecs, and Maps, and MapSpecs as well.

Is it possible to devise an approach that works with Variables initially, but can be extended for other object kinds later? Can we generate less code by using generics?

@smagnani96
Copy link
Contributor Author

It probably makes sense to land basic bpf2go variable support first:

  • VariableSpecs included in generated xxxSpecs; do we need to segregate them similar to today's xxxProgramSpecs/xxxMapSpecs?
  • Variables included in generated xxxObjects.

I see huge value in a type-safe get/set API you are proposing. It would be nice to have not only Variables covered, but probably VariableSpecs, and Maps, and MapSpecs as well.

Is it possible to devise an approach that works with Variables initially, but can be extended for other object kinds later? Can we generate less code by using generics?

Do you suggest to change this PR to provide basic VariableSpecs support in bpf2go or should I open a new one?

Concerning the type-safe API, I can keep exploring solutions. I tried with generics with no luck, but maybe there's still a way...AFAIU CollectionSpec would also need to be a generic holding maps and variables each of different generic types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

New API in Collection{Spec} for modifying global BPF variables
3 participants