diff --git a/_posts/2025-01-17-colour-to-alpha.md b/_posts/2025-01-17-colour-to-alpha.md new file mode 100644 index 0000000..228c695 --- /dev/null +++ b/_posts/2025-01-17-colour-to-alpha.md @@ -0,0 +1,129 @@ +--- +title: Replacing an image colour with transparency +layout: post +excerpt: What happens if you convert an RGB image to RGBA by pretending it was sitting on a white background? +draft: true + +images: /assets/blog/alpha_test +thumbnail: /assets/blog/alpha_test/thumbnail.png +social_image: /assets/blog/alpha_test/thumbnail.png + +# The alt text for both images. +alt: An image of Mixtela's latest project, a pendant with a fluid simulation running on a LED matrix. +image_class: no-dim +mathjax: true + +--- + +I was looking at [Mixtela's latest project][mixtelas_project] and admiring how nicely his images blend with the background of the page. He has a simple white background and his images all have perfect white backgrounds with just a little hint of a shadow. + +
+ +
An image of [Mixtela's fluid simulation pendant][mixtelas_project]. +
+
+ +I think he achieves this through by simply doing very good photography, he probably photographs the object under good lighting in a white booth type thing. I suspect he also adjusts the white balance in post because the white background pixels are all exactly `(255,255,255)`. + +But my site has a slightly off white background and it also has a dark mode. Is there some way I could make a similar image that adapts to the background colour? + +Well I can kinda think of a crude way. What if we tried to invert the alpha blending process to derive an RGBA image from an RGB image and a background colour? + +For a particular pixel of the image, the output pixel $c_{out}$ is just the linear combination of the background $b$ and foreground $f$ colours weighted by the alpha channel $\alpha$: + +$$ c_{\text{out}} = f \alpha + b (1 - \alpha) $$ + +I'm gonna fix the output colour $c_{\text{out}}$ to be the rgb colour of my source image and the background $b$ as white. This gives us: + +$$ f = \left( c_{\text{out}} - b (1 - \alpha) \right) / \alpha $$ + +Now we have to choose alpha for every pixel. Note that's it's not an entirely free choice, any pixel that isn't white in the source image has a maximum alpha we can set before we would start getting negative values in the solution. + +For white the maximum value of alpha turns out to be just the minimum of the r, g and b channels. For a different choice of background colour it would be the minimum of the three channels of $c_{\text{out}} / b$. + +Logically some parts of this image should not be transparent, the actual pendant itself is clearly made out of metal so you wouldn't be able to see through it. The shadow on the other hand would make sense as a grey colour with some transparency. + +However I'm just going to see what I get if I set alpha to the maximum possible value for each pixel. + +```python +import numpy as np +from PIL import Image +from pathlib import Path + +input_path = Path("pendant-complete1.jpg") + .expanduser() + +# convert to 64bit floats from 0 - 1 +color = np.asarray(Image.open(input_path) + .convert("RGB")) + .astype(np.float64) / 255.0 + +# The amount of white in each pixel +white = np.array([1.,1.,1.]) +alpha = 1 - np.min(color, axis = 2) + +premultiplied_new_color = color \ + - (1 - alpha)[:, :, None] \ + * white[None, None, :]) + +# This does new_color = premultiplied_new_color / alpha +# but outputs 0 when alpha = 0 +new_color = \ + np.divide( + premultiplied_new_color, + alpha[:, :, None], + out=np.zeros_like(premultiplied_new_color), + where = alpha[:, :, None]!=0 + ) + +new_RGBA = np.concatenate( + [new_color, alpha[:,:,None]], + axis = 2) + +img = Image.fromarray((new_RGBA * 255) + .astype(np.uint8), mode = "RGBA") +img.save("test.png") +``` + +And here are the results, switch the page to dark mode to see more of the effect. With a light, slightly off-white background the transparent image looks very similar to the original but now nicely blends into the background. + + + +Hit this button to switch to night mode: + + +
+ + + + +
Here are some images, (top left) original, (top right) white subtracted and replaced with alpha, (bottom left) same but brightened in dark mode, (bottom right) cutout based background removal tool (loses shadow)
+
+ +I quite like the effect, and because we chose to make all the pixels as transparent as possible, it has the added bonus that the image dims a bit in dark mode. + +## Addendum + +Harking back to my other post about Einstein summation notation, if we have in image with an index for height $$h$$ and width $$w$$ and colour channel $$c$$ that runs over `r,g,b`, we can write these equations as: + +$$ c_{\text{out}} = f_{hwc} \alpha_{hw} + b_{hw} (1 - \alpha_{hw}) \;\; \text{(No sum over} h, g)$$ + +so instead of +```python +premultiplied_new_color = color - \ + (1 - alpha)[:, :, None] * white[None, None, :] +``` + +we could also write: + +```python +premultiplied_new_color = color - np.einsum( + "xy, i -> xyi", (1 - alpha), white +) +``` + +...which is probably not that much simpler for this use case but when it becomes more helpful when you're not just doing elementwise operations and broadcasting. + +[mixtelas_project]: https://mitxela.com/projects/fluid-pendant \ No newline at end of file diff --git a/_posts/2025-01-18-heic-depth.md b/_posts/2025-01-18-heic-depth.md new file mode 100644 index 0000000..aeaabd2 --- /dev/null +++ b/_posts/2025-01-18-heic-depth.md @@ -0,0 +1,180 @@ +--- +title: Undexpected Depths +layout: post +excerpt: Did you know iPhone portrait mode HEIC files have a depth map in them? +draft: true +assets: /assets/blog/heic_depth_map + +thumbnail: /assets/blog/heic_depth_map/thumbnail.png +social_image: /assets/blog/heic_depth_map/thumbnail.png + +alt: An image of the text "{...}" to suggest the idea of a template. + +head: | + + + + + + +--- + +You know how iPhones do this fake depth of field effect where they blur the background? Did you know that the depth information used to do that effect is stored in the file? + +```python +# pip install pillow pillow-heif pypcd4 + +from PIL import Image, ImageFilter +from pillow_heif import HeifImagePlugin + +d = Path("wherever") + +img = Image.open(d / "test_image.heic") + +depth_im = img.info["depth_images"][0] +pil_depth_im = depth_im.to_pillow() +pil_depth_im.save(d / "depth.png") + +depth_array = np.asarray(depth_im) +rgb_rescaled = img.resize(depth_array.shape[::-1]) +rgb_rescaled.save(d / "rgb.png") +``` + +
+ + +
A lovely picture of my face and a depth map of it.
+
+ + +Crazy! I had a play with projecting this into 3D to see what it would look like. I was too lazy to look deeply into how this should be interpreted geometrically, so initially I just pretended the image was taken from infinitely far away and then eyeballed the units. The fact that this looks at all reasonable makes me wonder if the depths are somehow reprojected to match that assumption. Otherwise you'd need to also know the properties of the lense that was used to take the photo. + +This handy `pypcd4` python library made outputting the data quite easy and three.js has a module for displaying point cloud data. You can see that why writing numpy code I tend to scatter `print(f"{array.shape = }, {array.dtype = }")` liberally throughout, it just makes keeping track of those arrays so much easier. + +```python +from pypcd4 import PointCloud + +n, m = np_im.shape +aspect = n / m +x = np.linspace(0,2 * aspect,n) +y = np.linspace(0,2,m) + +rgb_points = np.array(rgb_rescaled).reshape(-1, 3) +print(f"{rgb_points.shape = }, {rgb_points.dtype = }") +rgb_packed = PointCloud.encode_rgb(rgb_points).reshape(-1, 1) +print(f"{rgb_packed.shape = }, {rgb_packed.dtype = }") + +print(np.min(np_im), np.max(np_im)) + +mesh = np.array(np.meshgrid(x, y, indexing='ij')) + +xy_points = mesh.reshape(2,-1).T +print(f"{xy_points.shape = }") + +z = np_im.reshape(-1, 1).astype(np.float64) / 255.0 + +m = pil_depth_im.info["metadata"] +range = m["d_max"] - m["d_min"] +z = range * z + m["d_min"] + +print(f"{xyz_points.shape = }") +xyz_rgb_points = np.concatenate([xy_points, z, rgb_packed], axis = -1) + +pc = PointCloud.from_xyzrgb_points(xyz_rgb_points) +pc.save(d / "pointcloud.pcd") +``` + +Click and drag to spin me around. It didn't really capture my nose very well, I guess this is more a foreground/background kinda thing. + + + + \ No newline at end of file diff --git a/_posts/2100-12-30-template.md b/_posts/2100-12-30-template.md index 59e4d2f..604b8a9 100644 --- a/_posts/2100-12-30-template.md +++ b/_posts/2100-12-30-template.md @@ -105,7 +105,11 @@ A table: -## Line Element +## Math + +Stack overflow has a nice [mathjax summary](https://math.meta.stackexchange.com/questions/5020/mathjax-basic-tutorial-and-quick-reference) + +List of mathjax symbols [here](https://docs.mathjax.org/en/latest/input/tex/macros/index.html) So the setup is this: Imagine we draw a very short line vector $\vec{v}$ and let it flow along in a fluid with velocity field $u(\vec{x}, t)$. @@ -166,6 +170,18 @@ _{T_{1}} _{T_{2}}. $$ +Aligning equations: + +$$ +\begin{align} +\sqrt{37} & = \sqrt{\frac{73^2-1}{12^2}} \\ + & = \sqrt{\frac{73^2}{12^2}\cdot\frac{73^2-1}{73^2}} \\ + & = \sqrt{\frac{73^2}{12^2}}\sqrt{\frac{73^2-1}{73^2}} \\ + & = \frac{73}{12}\sqrt{1 - \frac{1}{73^2}} \\ + & \approx \frac{73}{12}\left(1 - \frac{1}{2\cdot73^2}\right) +\end{align} +$$ + References: [This is a link to the subtitle heading at the top of the page](#subtitle) @@ -365,9 +381,10 @@ function animate() {
- - - - + + + +
Here are some images, (top left) original, (top right) white subtracted and replaced with alpha, (bottom left) same but brightened, (bottom right) ai background removal tool (loses shadow)
-
\ No newline at end of file + + diff --git a/_sass/base.scss b/_sass/base.scss index 4348eaf..fc2b938 100644 --- a/_sass/base.scss +++ b/_sass/base.scss @@ -36,6 +36,8 @@ --theme-highlight-color-transparent: hsla(338, 75%, 60%, 33%); --theme-subtle-text-color: #606984; + --night-mode-fade-time: 0.5s; + // constrain width and center --body-max-width: 900px; --body-width: min(100vw, 900px); @@ -233,7 +235,7 @@ figure.two-wide { justify-content: center; gap: 1em; margin-bottom: 1em; - *:not(figcaption) { + > *:not(figcaption) { width: calc(50% - 0.5em); } } @@ -244,7 +246,7 @@ figure.multiple { justify-content: center; gap: 1em; margin-bottom: 1em; - *:not(figcaption) { + > *:not(figcaption) { width: calc(50% - 0.5em); margin: 0; padding: 0; @@ -260,7 +262,7 @@ figure.multiple { margin-bottom: 1em; place-items: center center; - *:not(figcaption) { + > *:not(figcaption) { margin: 0; padding: 0; width: 100%; @@ -362,15 +364,16 @@ body:not(.has-wc) .has-wc { } // Add transitions for things that will be affected by night mode -body { - transition: background 500ms ease-in-out, color 200ms ease-in-out; +body, +a { + transition: background var(--night-mode-fade-time) ease-in-out, + color var(--night-mode-fade-time) ease-in-out; } -img { - transition: opacity 500ms ease-in-out; -} -svg.invertable, -img.invertable { - transition: filter 500ms ease-in-out; + +img, +svg { + transition: opacity var(--night-mode-fade-time) ease-in-out, + filter var(--night-mode-fade-time) ease-in-out; } @mixin night-mode { @@ -391,10 +394,15 @@ img.invertable { // Two main image classes are "invertable" i.e look good inverted // and "no-dim" i.e don't get dimmed in night mode // All other images get dimmed in night mode - img:not(.invertable):not(.no-dim) { + img:not(.invertable):not(.no-dim):not(.brighten) { opacity: 0.75; } + svg.brighten, + img.brighten { + filter: brightness(2); + } + svg.invertable, img.invertable { opacity: 1; diff --git a/_sass/cv.scss b/_sass/cv.scss index 49cfc1f..0b4cf53 100644 --- a/_sass/cv.scss +++ b/_sass/cv.scss @@ -43,7 +43,8 @@ summary.cv:before { left: 1rem; transform: rotate(0); transform-origin: 0.2rem 50%; - transition: 0.25s transform ease; + transition: 0.25s transform ease, + border-color var(--night-mode-fade-time) ease-in-out; } summary li { @@ -90,6 +91,8 @@ div.details-container { margin-top: 1em; border-bottom: var(--theme-subtle-outline) 1px solid; + transition: border-color var(--night-mode-fade-time) ease-in-out, + opacity var(--night-mode-fade-time) ease-in-out; h2 { margin: 0px; } diff --git a/_sass/header.scss b/_sass/header.scss index 72524d6..76997f8 100644 --- a/_sass/header.scss +++ b/_sass/header.scss @@ -34,6 +34,7 @@ header { border-radius: 50%; padding: 5px; border: 1px solid var(--theme-text-color); + transition: border-color var(--night-mode-fade-time) ease-in-out; } h1 { diff --git a/_sass/night_mode_toggle.scss b/_sass/night_mode_toggle.scss index c44b48a..d7023de 100644 --- a/_sass/night_mode_toggle.scss +++ b/_sass/night_mode_toggle.scss @@ -3,6 +3,7 @@ } .user-toggle { + display: inline; padding-top: 0.5rem; } @@ -17,7 +18,8 @@ color: var(--theme-text-color); background: var(--theme-background-color); border: 1.5px solid var(--theme-text-color); - transition: background 500ms ease-in-out, color 200ms ease; + transition: background var(--night-mode-fade-time) ease-in-out, + color var(--night-mode-fade-time) ease; } .toggle-button__icon { @@ -27,6 +29,6 @@ flex-shrink: 0; margin: 0; transform: translateY(0px); /* Optical adjustment */ - transition: filter 200ms ease-in-out; + transition: filter var(--night-mode-fade-time) ease-in-out; filter: var(--button-icon-filter); } diff --git a/_sass/thesis_styles.scss b/_sass/thesis_styles.scss index a82b323..ab53810 100644 --- a/_sass/thesis_styles.scss +++ b/_sass/thesis_styles.scss @@ -1,146 +1,148 @@ h1.thesis-title { - font-size: 3em !important; + font-size: 3em !important; } -main h1, h2, h3 { - font-family: "Source Serif Pro", serif; - font-weight: 300; - font-size: 2.2em !important; +main h1, +h2, +h3 { + font-family: "Source Serif Pro", serif; + font-weight: 300; + font-size: 2.2em !important; } // Make figures looks nice figure { - display: flex; - flex-direction: column; - align-items: center; - margin-inline-start: 0em; - margin-inline-end: 0em; - - max-width: 900px !important; + display: flex; + flex-direction: column; + align-items: center; + margin-inline-start: 0em; + margin-inline-end: 0em; - // border-bottom: solid #222 1px; - padding-bottom: 1em; + max-width: 900px !important; - // border-top: solid #222 1px; - // padding-top: 1em; + // border-bottom: solid #222 1px; + padding-bottom: 1em; + + // border-top: solid #222 1px; + // padding-top: 1em; } -figure > img, figure > svg { - // max-width: 90% !important; - margin-bottom: 2em; +figure > img, +figure > svg { + // max-width: 90% !important; + margin-bottom: 2em; } figcaption { - // font-style: italic; - // font-size: 0.9em; - max-width: 90%; + // font-style: italic; + // font-size: 0.9em; + max-width: 90%; } nav.page-table-of-contents > ul > li:first-child { - display: none; + display: none; } //For the animation that plays in the nav as you scroll nav.page-table-of-contents { - li li {font-size: 0.9em} + li li { + font-size: 0.9em; + } - ul { - padding-inline-start: 6px; - } + ul { + padding-inline-start: 6px; + } - a { - transition: all 200ms ease-in-out; - color: #000; - font-weight:normal; - } + a { + transition: all var(--night-mode-fade-time) ease-in-out; + color: #000; + font-weight: normal; + } - li.active > a { - color: #000!important; - font-weight:bold; - } + li.active > a { + color: #000 !important; + font-weight: bold; + } } // modify the spacing of the various levels li { - margin-bottom: 0.2em; + margin-bottom: 0.2em; } main > ul > li { - margin-top: 1em; + margin-top: 1em; } main > ul > ul > li { - margin-top: 0.5em; + margin-top: 0.5em; } // Pull the citations a little closer in to the previous word span.citation { - margin-left: -1em; + margin-left: -1em; - a { - text-decoration: none; - color: darkblue; - } + a { + text-decoration: none; + color: darkblue; + } } // Mess with the formatting of the bibliography div.csl-entry { - margin-bottom: 0.5em; + margin-bottom: 0.5em; } div.csl-entry a { -// text-decoration: none; - text-decoration: none; - color: darkblue; + // text-decoration: none; + text-decoration: none; + color: darkblue; } div.csl-entry div { - display: inline; + display: inline; } header li { - list-style: none; - a { - text-decoration: none; - margin-bottom: 0.5em; - display:block; - - } + list-style: none; + a { + text-decoration: none; + margin-bottom: 0.5em; + display: block; + } } nav.overall-table-of-contents > ul { - padding-inline-start: 0px; - + padding-inline-start: 0px; - > li { - list-style: none; - margin-top: 1em; - } + > li { + list-style: none; + margin-top: 1em; + } } // Page header div#page-header { - //make the header sticky, I don't really like how this looks but it's fun to play with - // position: sticky; - // top: 0px; - // background: white; - // z-index: 10; - // width: 100%; - p { margin-block-end: 0px;} + //make the header sticky, I don't really like how this looks but it's fun to play with + // position: sticky; + // top: 0px; + // background: white; + // z-index: 10; + // width: 100%; + p { + margin-block-end: 0px; + } } +@media only screen and (max-width: $horizontal_breakpoint), + only screen and (max-height: $vertical_breakpoint) { + //make the figures go to 100% and use italics to denote the figure captions + figure > img, + figure > svg { + max-width: 100% !important; + } -@media - only screen and (max-width: $horizontal_breakpoint), - only screen and (max-height: $vertical_breakpoint) - { - - //make the figures go to 100% and use italics to denote the figure captions - figure > img, figure > svg { - max-width: 100% !important; - } - - figcaption { - font-style: italic; - width: 100%; - } + figcaption { + font-style: italic; + width: 100%; + } } diff --git a/assets/images/alpha_test/ai_subtracted.png b/assets/blog/alpha_test/ai_subtracted.png similarity index 100% rename from assets/images/alpha_test/ai_subtracted.png rename to assets/blog/alpha_test/ai_subtracted.png diff --git a/assets/images/alpha_test/original.jpg b/assets/blog/alpha_test/original.jpg similarity index 100% rename from assets/images/alpha_test/original.jpg rename to assets/blog/alpha_test/original.jpg diff --git a/assets/blog/alpha_test/thumbnail.png b/assets/blog/alpha_test/thumbnail.png new file mode 100644 index 0000000..7ba3958 Binary files /dev/null and b/assets/blog/alpha_test/thumbnail.png differ diff --git a/assets/images/alpha_test/white_subtracted.png b/assets/blog/alpha_test/white_subtracted.png similarity index 100% rename from assets/images/alpha_test/white_subtracted.png rename to assets/blog/alpha_test/white_subtracted.png diff --git a/assets/blog/alpha_test/white_to_alpha.py b/assets/blog/alpha_test/white_to_alpha.py new file mode 100644 index 0000000..59a3a9f --- /dev/null +++ b/assets/blog/alpha_test/white_to_alpha.py @@ -0,0 +1,38 @@ +#!/usr/bin/env python3 +import sys + +import numpy as np +from PIL import Image + +if len(sys.argv) < 3: + print("Usage: python white_to_alpha.py ") + sys.exit(1) + +input_path, output_path = sys.argv[1], sys.argv[2] + +# convert to 64bit floats from 0 - 1 +d = np.asarray(Image.open(input_path).convert("RGBA")).astype(np.float64) / 255.0 + +#decompose channels +# r,g,b,a = d.T +color = d[:, :, :3] + +# The amount of white in each pixel +white = np.array([1.,1.,1.]) +white_amount = np.min(color, axis = 2) +alpha = 1 - white_amount + +premultiplied_new_color = (color - (1 - alpha)[:, :, None] * white[None, None, :]) +new_color = premultiplied_new_color / alpha[:, :, None] + +original_color = alpha[:,:,None] * new_color + (1 - alpha[:,:,None]) * white + +new_RGBA = np.concatenate([new_color, alpha[:,:,None]], axis = 2) + +# Premultiplied alpha, but PIL doesn't seem to support it +# new_RGBa = np.concatenate([premultiplied_new_color, alpha[:,:,None]], axis = 2) +# print(np.info(new_RGBA)) + +img = Image.fromarray((new_RGBA * 255).astype(np.uint8), mode = "RGBA") +img.save(output_path) +print(f"Image saved to {output_path}") \ No newline at end of file diff --git a/assets/blog/heic_depth_map/depth.png b/assets/blog/heic_depth_map/depth.png new file mode 100644 index 0000000..b7b7ff1 Binary files /dev/null and b/assets/blog/heic_depth_map/depth.png differ diff --git a/assets/blog/heic_depth_map/pointcloud.pcd b/assets/blog/heic_depth_map/pointcloud.pcd new file mode 100644 index 0000000..0b22261 Binary files /dev/null and b/assets/blog/heic_depth_map/pointcloud.pcd differ diff --git a/assets/blog/heic_depth_map/pointcloud_aperture.pcd b/assets/blog/heic_depth_map/pointcloud_aperture.pcd new file mode 100644 index 0000000..c5a5b7a Binary files /dev/null and b/assets/blog/heic_depth_map/pointcloud_aperture.pcd differ diff --git a/assets/blog/heic_depth_map/pointcloud_cylinder.pcd b/assets/blog/heic_depth_map/pointcloud_cylinder.pcd new file mode 100644 index 0000000..4c64a26 Binary files /dev/null and b/assets/blog/heic_depth_map/pointcloud_cylinder.pcd differ diff --git a/assets/blog/heic_depth_map/pointcloud_sphere.pcd b/assets/blog/heic_depth_map/pointcloud_sphere.pcd new file mode 100644 index 0000000..3d6b98f Binary files /dev/null and b/assets/blog/heic_depth_map/pointcloud_sphere.pcd differ diff --git a/assets/blog/heic_depth_map/rgb.png b/assets/blog/heic_depth_map/rgb.png new file mode 100644 index 0000000..c609ef5 Binary files /dev/null and b/assets/blog/heic_depth_map/rgb.png differ diff --git a/assets/blog/heic_depth_map/test_image.heic b/assets/blog/heic_depth_map/test_image.heic new file mode 100644 index 0000000..9abacd9 Binary files /dev/null and b/assets/blog/heic_depth_map/test_image.heic differ diff --git a/assets/blog/heic_depth_map/thumbnail.png b/assets/blog/heic_depth_map/thumbnail.png new file mode 100644 index 0000000..5660c19 Binary files /dev/null and b/assets/blog/heic_depth_map/thumbnail.png differ diff --git a/assets/blog/heic_depth_map/thumbnail.svg b/assets/blog/heic_depth_map/thumbnail.svg new file mode 100644 index 0000000..6ee5059 --- /dev/null +++ b/assets/blog/heic_depth_map/thumbnail.svg @@ -0,0 +1,9891 @@ + + + + diff --git a/assets/js/index.js b/assets/js/index.js index 7843e31..e335fd8 100644 --- a/assets/js/index.js +++ b/assets/js/index.js @@ -19,7 +19,7 @@ if (window.customElements) { document.querySelector("body").classList.add("has-wc"); } -const modeToggleButton = document.querySelector(".js-mode-toggle"); +const modeToggleButtons = document.querySelectorAll(".js-mode-toggle"); const modeStatusElement = document.querySelector(".js-mode-status"); const toggleSetting = () => { @@ -42,9 +42,10 @@ const toggleSetting = () => { localStorage.setItem(STORAGE_KEY, currentSetting); }; -modeToggleButton.addEventListener("click", (evt) => { - evt.preventDefault(); - - toggleSetting(); - applySetting(); +modeToggleButtons.forEach((m) => { + m.addEventListener("click", (evt) => { + evt.preventDefault(); + toggleSetting(); + applySetting(); + }); }); diff --git a/scripts/nice_point_cloud.pcd b/scripts/nice_point_cloud.pcd new file mode 100644 index 0000000..12e3223 Binary files /dev/null and b/scripts/nice_point_cloud.pcd differ