如何将一种类型的多个值合并为另一种类型的单个值

时间:2021-04-19 21:21:13

E.g given vector<uint8> of length 100, how to create new vector<uint16> of 50 elements. Preferably without copying?

E。g给定向量 的长度为100,如何创建新的向量 的50个元素。最好是没有复制吗?

(Edit: info from my comments)

(编辑:我的评论)

To illustrate:

说明:

I have a uint16 grayscale image file, my 3rd party lib returns a vector of uint8. Every 2 bytes = 1 pixel. It is practical for me to work with a vector of uint16. I think the only difference between this vector<uint8> and a corresponding vector<uint16> is that the bytes are read in a "concatenated" manner (i.e. chunks of 2 bytes = 1 value).

我有一个uint16灰度图像文件,我的第三方库返回一个uint8向量。每2个字节= 1个像素。对我来说,使用uint16向量是可行的。我认为这个向量 和对应的向量 的唯一区别是字节以“串联”的方式读取(即2字节的块= 1个值)。

I could loop and combine every 2 elements into a new vector element, but that seems inefficient, since the memory layout is the same. I was hoping I could combine some cast and maybe a move constructor to create a vector<uint16> --without copying the original vector<uint8> bytes again.

我可以将每两个元素循环并组合到一个新的向量元素中,但这似乎是低效的,因为内存布局是相同的。我希望我可以结合一些cast和一个move构造函数来创建一个向量 ——而不必再次复制原始向量 bytes。

Edit 2: To dispel any possible misunderstandings I drew a picture, forgive my poor ascii art :)

编辑2:为了消除可能出现的误解,我画了一幅图,请原谅我那蹩脚的ascii码:

container of uint8 values in memory:

内存中uint8值的容器:

[ _ ] | [ _ ] | [ _ ] | [ _ ] ...
|^^|
accessing element = accessing 1 byte

[_] |[_] |[_] |[_]…| ^ ^ | =访问1字节访问元素

container of uint16 values in memory is also just a sequence of bytes;

内存中uint16值的容器也是一个字节序列;

[ _ ] | [ _ ] | [ _ ] | [ _ ] ...

[_] |[_] |[_] |[_]…

|^ ^ ^ ^ ^|
accessing element = accessing 2 bytes (lets say my system read this as big-endian)

| ^ ^ ^ ^ ^ = |访问元素访问2字节(假设系统读取这是大端法)

I already have the sequence of bytes in vector v1. I just want a v2 of different type so I can access them differently. If it turns out the uint16 values are read as little-endian I could still work with that.

我已经有了v1中字节的序列。我只是想要一个不同类型的v2所以我可以以不同的方式访问它们。如果将uint16值改为little-endian,我仍然可以使用它。

Edit 3: So far it seems the answer by black is the best to my understanding (I will accept later if nothing changes), it still seems odd that there is no simple or STL solution to this. Though thanks to everyone for their prompt input and patience with my attempts at explaining myself.

编辑3:到目前为止,布莱克给出的答案似乎对我的理解是最好的(如果没有什么变化,我以后会接受的),但是没有简单或STL的解决方案似乎仍然是奇怪的。虽然感谢大家的及时输入和耐心,我尝试解释我自己。

4 个解决方案

#1


0  

vector<std::uint8_t> will hold a std::uint8_t* whilst vector<std::uint16_t> a std::uint16_t*. What you want to basically do is share the two pointers and give different interpretations, something like

向量 <:uint8_t> 持有std::uint8_t*,而向量 <:uint16_t> ::uint16_t*。你要做的就是分享这两个要点并给出不同的解释,比如

auto ptr = reinterpret_cast<std::uint16_t*>( vectorOfUint8.data() )

This is fine as long as you don't read/write through that pointer because such operation will cause undefined behavior due the strict-aliasing rule. You need to copy, which can be optimized to be very efficient with SIMD. If your compiler can't do it automatically, you can use intrinsics or unroll it:

只要您不通过该指针进行读/写操作,这是可以的,因为这种操作会由于严格别名规则而导致未定义的行为。您需要复制,这可以优化为使用SIMD非常有效。如果编译器不能自动执行,可以使用intrinsic或unroll:

  1. Read two elements from v1 to v2 so that they get 96 (48 in v2) manually
  2. 从v1到v2读取两个元素,这样它们就可以手工得到96 (v2中为48)
  3. Loop over v2 and read 4 (i += 48 / 4 = 12) or 8 (i += 48 / 8 = 6) elements at time
  4. 循环v2,并读取4 (i += 48 / 4 = 12)或8 (i += 48 / 8 = 6)元素

You could also disable the strict-aliasing rule, though that's compiler-specific and hence not standard and portable.

您还可以禁用严格别名规则,尽管这是特定于编译器的,因此不是标准的和可移植的。

#2


4  

As you don't control the source (per your comment), you can't know that the input vector has a 2-byte aligned buffer. For that reason alone, you have to copy the input vector.

由于您不控制源(根据您的注释),您无法知道输入向量有一个2字节对齐的缓冲区。仅仅因为这个原因,你必须复制输入向量。

How you do it won't matter much; memory access speed probably dominates the run time. However, do call reserve(50) on the destination vector - having multiple allocations will slow down the program.

你怎么做并不重要;内存访问速度可能控制运行时间。但是,要在目标向量上调用reserve(50)——多次分配会减慢程序的速度。

#3


1  

You can write a wrapper to do the conversion for you when needed. For example (without template)

您可以编写一个包装器,在需要时为您执行转换。例如(没有模板)

static inline uint16 getElement(const vector<uint8> &p, size_t index) {
  const int idx = index * 2;
  return p[idx] | p[idx + 1] << 8;
}

#4


0  

You would have to typecast, and I don't think you do it without copying in this case.

你必须进行排版,我认为在这种情况下你不需要复制就可以完成。

Vectors are a contiguous block of memory, and uint8 will result in all numbers occupying 8 bits and next to each other in memory. Now when you cast it to a uint16, every number will require an additional 8 bits, and you can't magically insert memory in-between a contiguous block. So, copying will occur.

向量是一个连续的内存块,uint8将导致所有数字占用8位,并且在内存中彼此相邻。现在,当您将它转换为uint16时,每个数字将需要额外的8位,并且您不能魔术般地在连续块之间插入内存。所以,复制就会发生。

Without the constraint of not copying, the problem is not hard, and I'm sure you can manage.

没有不复制的限制,这个问题并不难解决,我相信您可以做到。

EDIT: In response to the comment, even if the vector has space for 100 elements, but has less than 50 elements in it, typecasting will still require copying. The first two elements will occupy 16 bits, whereas later on the first element will be occupying those 16 bits. As a result, you have to at least copy the second element. In a similar logic, you will have to copy other elements.

编辑:对于注释,即使向量有100个元素的空间,但其中的元素少于50个,类型转换仍然需要复制。前两个元素将占用16位,而后一个元素将占用这16位。因此,至少要复制第二个元素。在类似的逻辑中,您将不得不复制其他元素。

#1


0  

vector<std::uint8_t> will hold a std::uint8_t* whilst vector<std::uint16_t> a std::uint16_t*. What you want to basically do is share the two pointers and give different interpretations, something like

向量 <:uint8_t> 持有std::uint8_t*,而向量 <:uint16_t> ::uint16_t*。你要做的就是分享这两个要点并给出不同的解释,比如

auto ptr = reinterpret_cast<std::uint16_t*>( vectorOfUint8.data() )

This is fine as long as you don't read/write through that pointer because such operation will cause undefined behavior due the strict-aliasing rule. You need to copy, which can be optimized to be very efficient with SIMD. If your compiler can't do it automatically, you can use intrinsics or unroll it:

只要您不通过该指针进行读/写操作,这是可以的,因为这种操作会由于严格别名规则而导致未定义的行为。您需要复制,这可以优化为使用SIMD非常有效。如果编译器不能自动执行,可以使用intrinsic或unroll:

  1. Read two elements from v1 to v2 so that they get 96 (48 in v2) manually
  2. 从v1到v2读取两个元素,这样它们就可以手工得到96 (v2中为48)
  3. Loop over v2 and read 4 (i += 48 / 4 = 12) or 8 (i += 48 / 8 = 6) elements at time
  4. 循环v2,并读取4 (i += 48 / 4 = 12)或8 (i += 48 / 8 = 6)元素

You could also disable the strict-aliasing rule, though that's compiler-specific and hence not standard and portable.

您还可以禁用严格别名规则,尽管这是特定于编译器的,因此不是标准的和可移植的。

#2


4  

As you don't control the source (per your comment), you can't know that the input vector has a 2-byte aligned buffer. For that reason alone, you have to copy the input vector.

由于您不控制源(根据您的注释),您无法知道输入向量有一个2字节对齐的缓冲区。仅仅因为这个原因,你必须复制输入向量。

How you do it won't matter much; memory access speed probably dominates the run time. However, do call reserve(50) on the destination vector - having multiple allocations will slow down the program.

你怎么做并不重要;内存访问速度可能控制运行时间。但是,要在目标向量上调用reserve(50)——多次分配会减慢程序的速度。

#3


1  

You can write a wrapper to do the conversion for you when needed. For example (without template)

您可以编写一个包装器,在需要时为您执行转换。例如(没有模板)

static inline uint16 getElement(const vector<uint8> &p, size_t index) {
  const int idx = index * 2;
  return p[idx] | p[idx + 1] << 8;
}

#4


0  

You would have to typecast, and I don't think you do it without copying in this case.

你必须进行排版,我认为在这种情况下你不需要复制就可以完成。

Vectors are a contiguous block of memory, and uint8 will result in all numbers occupying 8 bits and next to each other in memory. Now when you cast it to a uint16, every number will require an additional 8 bits, and you can't magically insert memory in-between a contiguous block. So, copying will occur.

向量是一个连续的内存块,uint8将导致所有数字占用8位,并且在内存中彼此相邻。现在,当您将它转换为uint16时,每个数字将需要额外的8位,并且您不能魔术般地在连续块之间插入内存。所以,复制就会发生。

Without the constraint of not copying, the problem is not hard, and I'm sure you can manage.

没有不复制的限制,这个问题并不难解决,我相信您可以做到。

EDIT: In response to the comment, even if the vector has space for 100 elements, but has less than 50 elements in it, typecasting will still require copying. The first two elements will occupy 16 bits, whereas later on the first element will be occupying those 16 bits. As a result, you have to at least copy the second element. In a similar logic, you will have to copy other elements.

编辑:对于注释,即使向量有100个元素的空间,但其中的元素少于50个,类型转换仍然需要复制。前两个元素将占用16位,而后一个元素将占用这16位。因此,至少要复制第二个元素。在类似的逻辑中,您将不得不复制其他元素。